-- Logs begin at Thu 2022-05-12 00:00:03 UTC, end at Thu 2022-05-12 01:43:09 UTC. -- May 12 00:38:20.123293 dv-cinder-830901-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 12 00:38:20.132346 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 12 00:38:20 2022] *** May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: nodename: dv-cinder-830901-5-iscsi May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: machine: x86_64 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: clock source: unix May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: pcre jit disabled May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: detected number of CPU cores: 4 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: current working directory: / May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: detected binary path: /usr/bin/uwsgi-core May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: your processes number limit is 47680 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: your memory page size is 4096 bytes May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: detected max file descriptor number: 2048 May 12 00:38:20.136964 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: lock engine: pthread robust mutexes May 12 00:38:20.137766 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: thunder lock: enabled May 12 00:38:20.137766 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 12 00:38:20.137766 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 12 00:38:20.183777 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: Python main interpreter initialized at 0x55d3e3bc89c0 May 12 00:38:20.183777 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: python threads support enabled May 12 00:38:20.183777 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: your server socket listen backlog is limited to 100 connections May 12 00:38:20.183777 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: your mercy for graceful operations on workers is 90 seconds May 12 00:38:20.184154 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: mapped 403077 bytes (393 KB) for 2 cores May 12 00:38:20.184279 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: *** Operational MODE: preforking *** May 12 00:38:20.184557 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: *** uWSGI is running in multiple interpreter mode *** May 12 00:38:20.184557 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: spawned uWSGI master process (pid: 96810) May 12 00:38:20.184762 dv-cinder-830901-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 12 00:38:20.186490 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: spawned uWSGI worker 1 (pid: 96825, cores: 1) May 12 00:38:20.186490 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: spawned uWSGI worker 2 (pid: 96826, cores: 1) May 12 00:38:20.186490 dv-cinder-830901-5-iscsi devstack@keystone.service[96810]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 12 00:38:25.814368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 00:38:25.814368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: from cryptography.utils import int_from_bytes May 12 00:38:25.815683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 00:38:25.815683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: from cryptography.utils import int_from_bytes May 12 00:38:25.819472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 00:38:25.819472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: from cryptography.utils import int_from_bytes May 12 00:38:25.820869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 00:38:25.820869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: from cryptography.utils import int_from_bytes May 12 00:38:25.864090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 12 00:38:25.864450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 12 00:38:25.864450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] command line args: [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 12 00:38:25.864700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 12 00:38:25.864919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ================================================================================ {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 12 00:38:25.865144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] admin_token = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.865409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] config_dir = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.865610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.865756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] config_source = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.865904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] debug = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.866136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] insecure_debug = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.868921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 12 00:38:25.868985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_config_append = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 12 00:38:25.869100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] command line args: [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 12 00:38:25.869232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_dir = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_file = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 12 00:38:25.869345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ================================================================================ {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 12 00:38:25.869453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] admin_token = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] config_dir = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] config_source = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.869945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] debug = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870136 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] insecure_debug = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_config_append = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.870907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.871024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.871087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_dir = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.871149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] notification_format = cadf {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.871206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_file = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.871854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] publish_errors = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.872964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.873162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.873347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.873347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.873754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.873754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.874103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.874389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.874669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.874842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.875078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.875248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] standard_threads = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.875436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.875627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] strict_password_check = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.875815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] use_eventlog = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] use_journal = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.876801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] use_json = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] notification_format = cadf {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] use_stderr = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] use_syslog = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] watch_log_file = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.877592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.877964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.878125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.878256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] publish_errors = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.878402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.878537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.878675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.878807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.878807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.879052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.879178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.879312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.879472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.879635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.879771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] standard_threads = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.879904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.880030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] strict_password_check = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.880550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.880689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.880815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.880940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] use_eventlog = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.881072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.881204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] use_journal = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.881204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] use_json = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.881454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.881585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] use_stderr = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.881711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.881836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] use_syslog = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.882042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] watch_log_file = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 00:38:25.882297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.882959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.883855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.884111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.884276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.884507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.884760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.884893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.885930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.886986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.887881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.888794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.889954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.890974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.891210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.891210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.891445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.891570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.892909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.893115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.893279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.893465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.893650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.893842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.894969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.895165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.895364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.895550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.895742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.895932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.896118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.896363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.896526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.896784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.896979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.897166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.897353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.897545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.897734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.897892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.898902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.899965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.900961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.enabled = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.901867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.902908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.903991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.904887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.905919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.enabled = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.907876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.908923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.external = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.mapped = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.909930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.password = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] auth.token = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] catalog.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.910965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.911889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.912977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.913881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.external = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.mapped = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.914943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.password = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] auth.token = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] catalog.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.915782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.916982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.917990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.918903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.919830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.920981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.921894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.922932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.923983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.924920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.925960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.926858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.927921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.928898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.password = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.929931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.930971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.931887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.932950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.933962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.password = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.934888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.935920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.936964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.937867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.938897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.939927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] policy.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.940894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] receipt.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941302 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.941941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.942959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.943898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] revoke.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.944950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] role.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] role.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] role.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] policy.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.945927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] role.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] receipt.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.946877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.947986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.948941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.949804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] revoke.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] role.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.950963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] role.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] role.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] role.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.951988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.952940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.953979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.954820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.provider = fernet {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.955976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.956964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] trust.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.957882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.958784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.connection = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.959989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.960916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.provider = fernet {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.961952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.962928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] trust.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.963977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.964962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=96825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.connection = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.965090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.966148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.967276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.967276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.967276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 00:38:25.967276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=96826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 12 00:38:25.978840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.979076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.979263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.979414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.979549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.979718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.987979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:25.988493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.017026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.017287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.017452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.017589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.074066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.074358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.125572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.125952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.128758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.128967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.192474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.192775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.192949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.193147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.193366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.193573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.193662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.193850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.194026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.194245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.194462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.194671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.217966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.218678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.230190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.230781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.245959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=96825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.246762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=96826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 00:38:26.264743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.264983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 00:38:26.265090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.265275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 00:38:26.265358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.265518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 00:38:26.265608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.265771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.265862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 00:38:26.266032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.266746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.277676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.277888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.277980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.278171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.278255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.278435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.278519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.278728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.286551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.286836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.289826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.290116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.290234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.290454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.290543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.290723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.290805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.291929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.292896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.319212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.319480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.320883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.321142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.321323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.321481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.326749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.326950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.327116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.327262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.327403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.327542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.334544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.334775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.334942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.335085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.339624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.339830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.339983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.340893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.341886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.360439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.360716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.361126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.361388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.365271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.366326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.369060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.369265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.370000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.370298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.374137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.375261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.377716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.378853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.381802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.382099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.382384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.382671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.382964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.383116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.383259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.383396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.383531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.383791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.384139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.384444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.402441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.402815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.403060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.403352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.403551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.403745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.404036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.404283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.411169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.412168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.412927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.413214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.413924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.414186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.416425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.416709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.416997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.417277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.417476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.417654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.417798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.417935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.418178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.418432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.431046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.431305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.431616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.431908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.432121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.432319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.432466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.432604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.432740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.432888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.433034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.433034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.433292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.433549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.450624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.453621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.453894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.456946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.457210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.458821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.460381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.460474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.460737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.461028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.461955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.463695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.463984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.464285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.468536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.470145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.471710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.473328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.473414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.473650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.473935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.474209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.476518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.476808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.477087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.477359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.483471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.483770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.484050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.486549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.486836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.487114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.491941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=96825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.492220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.492582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.492862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.493137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.493467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.493745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.494023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.494291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.494557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.494841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.495127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.495315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=96826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 00:38:26.495399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.495609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.495748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=96825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.495918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.496123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.496415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.496698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.496965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.497237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.497501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.498031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.498313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.498616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.498859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.499145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=96826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 00:38:26.535285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d3e3bc89c0 pid: 96825 (default app) May 12 00:38:26.538729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283305-dfcf-443a-aee2-41a9d98fba74 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:26.539033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283305-dfcf-443a-aee2-41a9d98fba74 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:26.539319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283305-dfcf-443a-aee2-41a9d98fba74 None None] PATH_INFO: `/v3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:26.539639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d3e3bc89c0 pid: 96826 (default app) May 12 00:38:26.543666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 12 00:38:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:34.531947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:34.532542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:34.532542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:34.536847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:34.542872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:34.543344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:34.543516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:34.575686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG oslo_db.sqlalchemy.engines [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=96825) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 12 00:38:34.763560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG passlib.handlers.bcrypt [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=96825) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 12 00:38:34.763986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG passlib.handlers.bcrypt [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=96825) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 12 00:38:34.820171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:34.822332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:34.956922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 12 00:38:35.144095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:35.181522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG oslo_db.sqlalchemy.engines [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=96826) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 12 00:38:35.310943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda66dc-d00d-4ef5-b0e3-8c4ba90d1317 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:35.315707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:35.316397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:35.316685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:35.316958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:35.325672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:35.379016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING keystone.common.rbac_enforcer.enforcer [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 12 00:38:35.379337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91687083-b288-4da7-a49a-993742f32d25 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:35.391101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 12 00:38:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:38:35.397376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:35.412041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b4a44-4a28-4771-be55-fbddf390ba8b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:35.417993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:35.419130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:35.419412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:35.419686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] PATH_INFO: `/v3/projects/137a089466c04c34b8c9ec48a39d84b0` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:35.421378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] RBAC: Authorizing `identity:get_project(project_id=137a089466c04c34b8c9ec48a39d84b0)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:35.472108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING keystone.common.rbac_enforcer.enforcer [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 12 00:38:35.472471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2602d-95dc-44cd-9386-30ec5538c848 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:35.477093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 12 00:38:35 2022] GET /identity/v3/projects/137a089466c04c34b8c9ec48a39d84b0 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:38:37.638753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79682-9c48-4de0-9c51-78f2f3ab225e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:37.639290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79682-9c48-4de0-9c51-78f2f3ab225e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:37.639290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79682-9c48-4de0-9c51-78f2f3ab225e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:37.643490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:38:37.650611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:37.651307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:37.651461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:37.677721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:37.679250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:37.688721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 12 00:38:37.710686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a79682-9c48-4de0-9c51-78f2f3ab225e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:37.724794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a79682-9c48-4de0-9c51-78f2f3ab225e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:37.730058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:37.730680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:37.730963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:37.731239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:37.738933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:37.740739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783eee3-e2d7-4c61-9d01-30bfaeb8457f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:37.749675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 12 00:38:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:38:37.756123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:37.767281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b860f44-f1da-4e77-a4dc-a22fa1973276 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:37.772863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:37.773722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:37.774005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:37.774276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:37.776066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:37.777916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fdf77-ef63-403d-853a-cceec0070c54 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:37.779466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 12 00:38:37 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:40.091778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0c32-25b1-40d7-ac93-6aa89a223058 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.092406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0c32-25b1-40d7-ac93-6aa89a223058 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.092406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0c32-25b1-40d7-ac93-6aa89a223058 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.093714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:40.099066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.099486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.099582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.110601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a5f-fc6e-4b79-8680-ce4b42559336 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.110884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a5f-fc6e-4b79-8680-ce4b42559336 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.111156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a5f-fc6e-4b79-8680-ce4b42559336 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.112543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:38:40.118869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.119291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.119581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.162409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG passlib.handlers.bcrypt [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=96826) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 12 00:38:40.162862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG passlib.handlers.bcrypt [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=96826) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 12 00:38:40.210161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:40.211782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:40.231796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:40.233021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:40.314944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:40.324647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:38:40.337114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.347952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.439659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1fba3-61b3-4108-bfd1-59cf44d37d12 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.449647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c90f-6a2e-4dc3-b8d6-cb6829f032b5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.478452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.479129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.479414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.479692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.485731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.486794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.487079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.487354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.508104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:40.519565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:40.520518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:38:40.521325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 8/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:40.527091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.547992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:40.549520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:40.629102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798c57-7582-403e-bbb7-9b43c7c039ff None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.647997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 7/15] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:38:40.654230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.759024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201241b1-5b58-4471-b351-507ad8bc3792 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.777171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.777843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.778128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.778409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.779163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:40.779470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:40.780952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:40.790868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.791664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.791942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.792215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.792923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:40.798659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.802563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:40.804023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:38:40.824499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:40.825348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:40.826129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:38:40.826129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:38:40.826129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:38:40.826129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:38:40.826129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 8/17] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:38:40.831668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.900503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7949b5b-7244-471a-99be-6e8c75a8a160 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.930953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0535-dacc-4aa6-80df-011ec5132504 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:40.941046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.942299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.942578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.942873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.944867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:40.946685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:40.948349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:38:40.953918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:40.966584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:40.967244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:40.967524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:40.967802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:40.968554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:40.968856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:40.970246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:40.986873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:41.065691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a099aff-388b-41e3-92d9-207bbcb83665 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:41.112312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:41.113170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:41.113455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:41.113734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:41.115037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:41.116569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530fb93-1320-4c72-97f2-d12a68447905 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:41.123478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:41.124633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:41.232184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40726e-a388-4d9a-824b-38f19df6fd6f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:41.271032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:41.276205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:41.283355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:41.283661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:41.284671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.domain.id=default, user.id=e279c42792694abda4ec83a94b9d0a59) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:41.284936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:41.286454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78497-e514-485a-a00c-52b6c4f2eca7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:41.294318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 12 00:38:41 2022] GET /identity/v3/role_assignments?user.id=e279c42792694abda4ec83a94b9d0a59&scope.domain.id=default&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:44.042785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1f81-b5a7-49a5-8080-566bcd548b0b None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.042785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1f81-b5a7-49a5-8080-566bcd548b0b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.042785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1f81-b5a7-49a5-8080-566bcd548b0b None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.043986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:44.050520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101dda7-1a70-4cc9-8c73-7afd8d6d3d05 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.051034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101dda7-1a70-4cc9-8c73-7afd8d6d3d05 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.051130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101dda7-1a70-4cc9-8c73-7afd8d6d3d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.072939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4c8-5938-4eac-aea1-520856c8e3e2 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.074630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4c8-5938-4eac-aea1-520856c8e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.075106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4c8-5938-4eac-aea1-520856c8e3e2 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.075640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7101dda7-1a70-4cc9-8c73-7afd8d6d3d05 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:44.076912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7101dda7-1a70-4cc9-8c73-7afd8d6d3d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:44.081569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 13/23] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:44.083260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.083744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.084223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.112765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:44.114405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:44.168984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 11/24] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:44.172289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b20e-c7c3-49e1-a696-0b42672ffe7f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.174715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b20e-c7c3-49e1-a696-0b42672ffe7f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.175202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b20e-c7c3-49e1-a696-0b42672ffe7f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.180749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 12/25] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:44.185790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.186301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.186767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.216959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:44.218130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:44.229978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 14/26] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:44.230867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:44.316727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 13/27] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:38:44.317540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:44.435609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023d34-7a40-4207-8b9f-61fa4d98d36b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:44.466464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87c8a-73a8-49f2-b4ca-a45ee2f9a007 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:44.471406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:44.472213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.472523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.472800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.482925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:44.484430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:38:44.503104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:44.503946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:44.505582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:38:44.505582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:38:44.505582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:38:44.505582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:38:44.505582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 14/28] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:44.505840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:44.516671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:44.517611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.517894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.518216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.531618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-bf2c4001-4682-405b-aefa-a19a527dd59a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:44.544469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:44.545317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:44.546083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:38:44.547356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 15/29] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:44.549203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39d90-3a62-44d4-9180-2f36ff3de1a0 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.549203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39d90-3a62-44d4-9180-2f36ff3de1a0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.549203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39d90-3a62-44d4-9180-2f36ff3de1a0 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.556365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 16/30] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:44.557866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39d90-3a62-44d4-9180-2f36ff3de1a0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:44.614345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1f1df-fb08-4999-b6f2-59fabcea461a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:44.649355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:44.650044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.650432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.650711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.651470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:44.652904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4209f-c25d-4004-8ae2-9c623b54b559 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:44.704732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39d90-3a62-44d4-9180-2f36ff3de1a0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:44.739295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:44.740383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.740841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.741286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.742183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:44.742682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:44.744416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:44.766227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:44.767450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:44.783663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 15/32] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/roles => generated 219 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:44.793250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:44.793581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:44.793808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:44.825480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:44.827111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:44.936783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f241ddd-f8ca-43c5-80fb-a2ce75ba4d9a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:44.965125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 16/33] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:44.966002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.023187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.023937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.024271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.024577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.025413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:45.025723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.027155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.048353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 18/34] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:45.049225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.114851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd43a5-fd33-4f6c-8cbd-39f35720d1e2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.171655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.172669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.173071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.173461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.174531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.177060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.213972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127d18-30f1-4dd6-aa0b-56f8b41d3108 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.224044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 17/35] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:38:44 2022] POST /identity/v3/projects => generated 301 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:45.225515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.294495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.295653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.296662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.296946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.297734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.299196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4624347-ef86-44a2-b118-c82ef70773f7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.395516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397cf2-a978-4299-9367-b09defbd5947 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.435354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.436292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.436576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.436859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.438522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.440381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094a2b-11d6-4eb9-8ebf-626bd596f606 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.446979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 18/36] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:45 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:45.446979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9cd15-6029-41ec-a359-8a37979b18ec None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.447226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 19/37] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 12 00:38:45 2022] POST /identity/v3/roles => generated 225 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:45.447226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4d7c-6052-464b-9cb7-3c910da02b5c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.447226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4d7c-6052-464b-9cb7-3c910da02b5c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.447226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4d7c-6052-464b-9cb7-3c910da02b5c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.447985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 20/38] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:45.449146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e4d7c-6052-464b-9cb7-3c910da02b5c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.454481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9cd15-6029-41ec-a359-8a37979b18ec None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.454481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9cd15-6029-41ec-a359-8a37979b18ec None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.454481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 19/39] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:45.455815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886cb4-a1a2-4c26-9ed9-749de9639287 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.456105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886cb4-a1a2-4c26-9ed9-749de9639287 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.456421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886cb4-a1a2-4c26-9ed9-749de9639287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.483725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d6886cb4-a1a2-4c26-9ed9-749de9639287 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:45.484928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d6886cb4-a1a2-4c26-9ed9-749de9639287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:45.584285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e4d7c-6052-464b-9cb7-3c910da02b5c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.584481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 20/40] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:45.584481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.584481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.584481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.595802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:45.597243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:45.620391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.621504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.621792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.622064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.624530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.627717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.631132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 21/41] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:38:45.631936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.695750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 21/42] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:45.695750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.771609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943e9d-c173-499f-a65e-5dd8f9fd9fb0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.799569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ff4f-1aed-4844-858c-1328b179a8fe None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.807828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.808536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.808826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.809111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.809874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.811292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.825928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 22/43] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 12 00:38:45 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:45.826738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:45.838303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.841018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.842079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.842761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] PATH_INFO: `/v3/domains/default/users/e279c42792694abda4ec83a94b9d0a59/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.846240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e279c42792694abda4ec83a94b9d0a59, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.848939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.873478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 22/44] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 12 00:38:45 2022] PUT /identity/v3/domains/default/users/e279c42792694abda4ec83a94b9d0a59/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:38:45.944871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09a080-147d-478c-a229-2776ef1d08f5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:45.983919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:45.984783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:45.985063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:45.985343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:45.986665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:45.988184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e80a9-f50f-4557-b1fc-adc123508d1f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:45.990212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:45.996529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:46.113342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf455a8-dc0b-4c4e-b351-ac2f607e09f8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:46.155695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:46.156506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:46.156678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:46.156954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:46.157664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:46.159128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082f9c-674f-4665-b82b-25d2b18f123d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:46.185821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:38:45 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:48.186168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362451b9-73bd-4046-9cf7-163d4796879e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.186689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362451b9-73bd-4046-9cf7-163d4796879e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.186689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362451b9-73bd-4046-9cf7-163d4796879e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.188112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:48.193477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d6cc-8fd9-4f62-8069-61b3a513aca7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.193915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d6cc-8fd9-4f62-8069-61b3a513aca7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.194002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d6cc-8fd9-4f62-8069-61b3a513aca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.211804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-daa6d6cc-8fd9-4f62-8069-61b3a513aca7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:48.213160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-daa6d6cc-8fd9-4f62-8069-61b3a513aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:48.288898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:48.312762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362451b9-73bd-4046-9cf7-163d4796879e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.409363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2cff-d8fc-467e-9ebd-69ea1fa30f5e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.409576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2cff-d8fc-467e-9ebd-69ea1fa30f5e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.409853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2cff-d8fc-467e-9ebd-69ea1fa30f5e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.411282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 25/49] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:48.416797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.417179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.417540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.418743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362451b9-73bd-4046-9cf7-163d4796879e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.435541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:48.436791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:48.453895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.454703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.454999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.455279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.465868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:48.473492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:48.474354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:38:48.475106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 25/50] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:48.480756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.501554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 26/51] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:48.524957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.591902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f0cc-41b0-432b-99e4-f068cfadf461 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.624673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.625350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.625635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.625909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.626661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:48.627016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:48.628427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:48.628535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6deeb-40a2-4e2c-afce-ddf21e23d234 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.637275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 26/52] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:48.642725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.661738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.662464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.662742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.663017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.664196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:48.665714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:48.667314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:48.672792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.738178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3eeb-8793-433e-9e5d-837e0ce7dc7a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.766108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1967fca-49f2-4104-bc74-533ce1684f71 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.770535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.771431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.771719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.772002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.773308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:48.775045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:48.776776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 27/54] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:48.782163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:48.798067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.798731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.799010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.799289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.799990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:48.801476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:48.827585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 28/55] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 12 00:38:48 2022] POST /identity/v3/projects => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:38:48.887085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4fee9-a349-4a64-a6a9-252ec30a3962 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:48.922936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:48.923657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:48.923936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:48.924211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:48.925453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:48.926915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8026a-3041-485b-b06e-603d299083f2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:48.928609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 28/56] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:48.935194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:49.045915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54febe66-9884-4864-bbfc-88156e3c2d4c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:49.080319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:49.081114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:49.081396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:49.081672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:49.082512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.domain.id=default, user.id=e279c42792694abda4ec83a94b9d0a59) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:49.082832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:49.084319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd45ead-8ac0-44fd-9c9a-e8386b868039 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:49.090713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 29/57] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 12 00:38:48 2022] GET /identity/v3/role_assignments?user.id=e279c42792694abda4ec83a94b9d0a59&scope.domain.id=default&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:51.101760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1d3-70f9-4252-a730-c90e5b0c46f0 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:51.102366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1d3-70f9-4252-a730-c90e5b0c46f0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:51.102366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1d3-70f9-4252-a730-c90e5b0c46f0 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:51.103752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 29/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:51.110078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:51.110495 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:51.110692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:51.137585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:51.138784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:51.206531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 30/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:51.238787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb1d3-70f9-4252-a730-c90e5b0c46f0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:51.349713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb1d3-70f9-4252-a730-c90e5b0c46f0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:51.383913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:51.384784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:51.385064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:51.385338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:51.386580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:51.388028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16625a2d-f27f-41b7-86d9-d53a2d9a91ad None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:51.389650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 30/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:51.395591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:51.501478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe049-6191-45cd-8c8c-21f7f66475f1 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:51.542021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:51.543041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:51.543551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:51.543990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:51.545192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:51.547540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64b12-42f3-4b24-abfc-b1d43d7cbce7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:51.577211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 31/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:38:51 2022] POST /identity/v3/users => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:55.069423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509a9f-7aab-4044-b738-7fbc32c735c8 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.069423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509a9f-7aab-4044-b738-7fbc32c735c8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.069423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509a9f-7aab-4044-b738-7fbc32c735c8 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.069423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 31/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:55.079152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.079626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.079727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.105530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:55.106793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:55.148273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af29f4-561b-436f-a066-34d8f1ee5771 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.148273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af29f4-561b-436f-a066-34d8f1ee5771 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.148273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af29f4-561b-436f-a066-34d8f1ee5771 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.149691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 32/63] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:38:55.155011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b544d6-a36f-434a-87bb-c0b59c341ebd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.155535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b544d6-a36f-434a-87bb-c0b59c341ebd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.156001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b544d6-a36f-434a-87bb-c0b59c341ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.185343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a6b544d6-a36f-434a-87bb-c0b59c341ebd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:55.186922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a6b544d6-a36f-434a-87bb-c0b59c341ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:55.216368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 32/64] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:55.281869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.282805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 33/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:38:55.290454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542ac98-a8a1-45e6-8e47-33e8da268562 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.290975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542ac98-a8a1-45e6-8e47-33e8da268562 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.291387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542ac98-a8a1-45e6-8e47-33e8da268562 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.293025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 34/66] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:38:55.298380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.298863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.299272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.330247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:55.331651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:55.393009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51809f4-5f39-4db8-8a0c-b27b41bce4af None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.399223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 35/67] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:55.400625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.447460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.448319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.448606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.448887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.460756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:55.469784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:55.470731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:38:55.471512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:38:55.475911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 33/68] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:55.476959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.507449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8219f6-b4c9-41f6-8ae2-d55bb0fcd066 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.545699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.547296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.547926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.548504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.561405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:55.569686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:55.570815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:38:55.571734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:38:55.573888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 36/69] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:55.574867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.582756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f9257-dcde-4d34-9096-3aee9473e0b4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.616669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.617619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.617784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.618069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.628230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:55.637112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:55.638121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:38:55.639069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:38:55.643147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 34/70] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:55.643874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.678483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc590b-fb9e-4091-ab64-aff885fa95fa None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.714177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.714853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.715134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.715407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.716152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:55.716482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:55.718305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7113f-eaa6-47f8-b7a9-2e876b609e78 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:55.727080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 37/71] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:55.728234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662c83-8e06-4d5b-a811-a2fd80aeca81 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.728787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662c83-8e06-4d5b-a811-a2fd80aeca81 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.728891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662c83-8e06-4d5b-a811-a2fd80aeca81 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.730399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 38/72] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:55.730965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662c83-8e06-4d5b-a811-a2fd80aeca81 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.748043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16502435-faa9-4b68-aa42-ca5f305c3a09 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.780065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.780763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.781046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.781324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.782075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:55.782382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:55.783784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:55.795426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 35/73] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:55.796223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.830849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662c83-8e06-4d5b-a811-a2fd80aeca81 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.863774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.864477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.864763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.865038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.865792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:55.866101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:55.867496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6687ba5-a35d-4520-b471-58e745abb2cf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:55.875175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 39/74] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:55.876625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f5dc-7165-450e-ae74-56c63fdf0bff None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.876918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f5dc-7165-450e-ae74-56c63fdf0bff None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.877201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f5dc-7165-450e-ae74-56c63fdf0bff None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.878435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 40/75] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:38:55.881451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.881739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.882004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.897964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:55.898876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3031370-fc0c-4f43-bd6d-afd7e53cf87d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:55.899144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:55.933756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:55.934636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:55.934917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:55.935190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:55.936464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:55.937927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:55.939559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 36/76] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:55.940346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:55.961187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 41/77] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:55.962032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.042011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5a3a-4028-447f-8e1c-5f3ecbda4727 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.053754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3af31-335e-4f5b-aa72-507fd17a553c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.075517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.076429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.076712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.077000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.085228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.086102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.086378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.086648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.090619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.092099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c84661-075e-474b-b5a9-7b25e10e72cf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.093791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 37/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:56.094991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.095279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.095545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.101974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.103416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.105161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 42/79] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:55 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:56.109109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.112791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:38:56.114042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:38:56.179193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 38/80] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:38:56.180060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.209888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c9e2-966e-4a03-b74a-c0829f4323ce None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.244146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.244860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.245140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.245415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.277361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a5e6-bd6e-4829-9fd3-5362d10e3ffa None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.311844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.312654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.312932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.313025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.313209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.314413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.314509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.315868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.317523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 39/81] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:56.318296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.410680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 43/82] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:38:56.411452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.411725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004275a3-ac1d-4b46-bf7c-a99332de0b71 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.450473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.451135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.451411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.451692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.514671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4e946-654d-4789-a2e0-0b8583e88528 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.522530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.524036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.546202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.546840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.547127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.547402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.611196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.612680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.619824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 40/83] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:38:56.620733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.710430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 44/84] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:38:56.711184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.711483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21d58b-b120-49dd-9f00-8832264101ab None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.745862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.746682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.746965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.747237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.757827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:38:56.765630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:38:56.766633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:38:56.767467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 41/85] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:38:56.768181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.811013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd83c3-8d06-4b4f-9bae-41cf2234bf91 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.844595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.845271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.845553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.845829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.846541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.847983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.864493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e08aea-3b6e-4f8b-aaa4-def40d5c782e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:56.875820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 45/86] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:38:56 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:38:56.876683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.906759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:56.907720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:56.908006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:56.908320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:56.909730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:56.911330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:56.913343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 42/87] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:56.913959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:56.986939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622f0b6-363f-406c-8142-fab0f4e81157 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.023068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.024011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.024319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.024601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.025907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91881-e247-49e6-acae-268b66baf99b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.026051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.027632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.034311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 46/88] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.034311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.064909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.065781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.066068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.067148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.067552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.069579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.071333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 43/89] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:38:56 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.072159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.147446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a65e2-ec30-441b-9eb0-48c90ada79bb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.191200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.192170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.192762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.193226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.193998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:57.194280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.195945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.203025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be00a2-6e2d-4175-baa7-e67df6b1a909 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.205785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 47/90] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.206635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.245866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.246354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.246643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.246926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.248009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=e279c42792694abda4ec83a94b9d0a59) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:57.248110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.249649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.258224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 44/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/role_assignments?user.id=e279c42792694abda4ec83a94b9d0a59&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.259114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.333209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57723bf8-a489-4f19-8e2d-2c7d20346910 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.376037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.381404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.381788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.382072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.382989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:57.383307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.385192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.396357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 48/92] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.397311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.422399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c660-7608-4c1f-a32d-04381cdadf31 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.506513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.507668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.508163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.508674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.510210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:57.512604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.514419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.531235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 45/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.612028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932129a8-6ef3-4689-8cf5-3897866e9878 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.674104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.675004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.675286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.675561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.677071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.678568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:57.680276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 49/94] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:57.686545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:57.831616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bc221-46b2-4034-8849-ee824e506ec7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:57.864876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:57.865531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:57.865816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:57.866097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:57.938784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:57.940350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:58.057884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 46/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:38:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:38:58.063616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:58.192821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37c5f0-c8cc-4d8b-8fb1-9eaee7d0b9e5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:58.231707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:58.232606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:58.232889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:58.233162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] PATH_INFO: `/v3/projects/0b284590702d411ebd823f81b2ce993f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:58.234345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] RBAC: Authorizing `identity:get_project(project_id=0b284590702d411ebd823f81b2ce993f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:58.235794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36a8f5-d5a2-42cf-b941-1a29612ae44e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:58.237520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 50/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:38:58 2022] GET /identity/v3/projects/0b284590702d411ebd823f81b2ce993f => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:38:58.245734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:38:58.394455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa6f1-db63-4575-9afc-0f5bc26b6165 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:38:58.435850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:38:58.436460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:38:58.436743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:38:58.437021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:38:58.437867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=0b284590702d411ebd823f81b2ce993f, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:38:58.438181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:38:58.439635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd6862-043e-4e3b-be0d-22f54e2db549 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:38:58.445605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 47/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:38:58 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=0b284590702d411ebd823f81b2ce993f&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:00.100131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28184512-b6ba-42be-a5af-7248c378b144 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.100713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28184512-b6ba-42be-a5af-7248c378b144 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.100713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28184512-b6ba-42be-a5af-7248c378b144 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.102220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 51/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:00.110064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.110578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.110578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.131142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:00.133018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:00.202521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 48/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:00.225675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28184512-b6ba-42be-a5af-7248c378b144 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:00.329683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28184512-b6ba-42be-a5af-7248c378b144 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:00.367276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:00.368102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.368423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.368708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.380866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-1164e90d-57b9-4cd9-98e9-109e85addd1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:00.389859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:00.390827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:00.391617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:00.393113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 52/100] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:00.398951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:00.416752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e346fee-dffd-4cc7-91f9-de9b92dcf644 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.417069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e346fee-dffd-4cc7-91f9-de9b92dcf644 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.417310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e346fee-dffd-4cc7-91f9-de9b92dcf644 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.418883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 53/101] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:00.423723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.424021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.424325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.457280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:00.458502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:00.537219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 54/102] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:00.565029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:00.599161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279759f-ac91-47c1-8643-be38ec71a591 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:00.686526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:00.687569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.688061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.688598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.689579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:00.690093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:00.691726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4468-b0ff-418f-a75b-707df4ee2464 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:00.697738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3403ec-34ac-4362-b0c6-4c08953c59b3 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:00.710812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 49/103] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:00.712505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1232a0-c572-4cb7-9fbc-396c0e484b73 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.713078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1232a0-c572-4cb7-9fbc-396c0e484b73 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.713557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1232a0-c572-4cb7-9fbc-396c0e484b73 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.718581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 50/104] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:00.719672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1232a0-c572-4cb7-9fbc-396c0e484b73 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:00.742828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:00.742828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.742828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.742828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.744200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:00.746387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bb8a0-9d70-4984-8597-6576e3136f11 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:00.748196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 55/105] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:00.753449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5887ef-ba77-447c-9512-8afa46282dd9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.753738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5887ef-ba77-447c-9512-8afa46282dd9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.754021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5887ef-ba77-447c-9512-8afa46282dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.776620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0d5887ef-ba77-447c-9512-8afa46282dd9 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:00.779103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0d5887ef-ba77-447c-9512-8afa46282dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:00.841643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1232a0-c572-4cb7-9fbc-396c0e484b73 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:00.866404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 56/106] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:00.868972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8250-03ab-4bfb-9120-daa030689b56 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.869523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8250-03ab-4bfb-9120-daa030689b56 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.870230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8250-03ab-4bfb-9120-daa030689b56 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.872678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 57/107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:00.873965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf8250-03ab-4bfb-9120-daa030689b56 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:00.884413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:00.885794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.885794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.886101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] PATH_INFO: `/v3/users/e8639cfb66ff484fa419049d0399c82c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.904989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] RBAC: Authorizing `identity:get_user(user_id=e8639cfb66ff484fa419049d0399c82c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:00.907618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98141231-d936-4ecb-b5c3-963aa1d03604 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:00.910626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 51/108] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:00 2022] GET /identity/v3/users/e8639cfb66ff484fa419049d0399c82c => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:00.916288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:00.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:00.917177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:00.938296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:00.940539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:00.981620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf8250-03ab-4bfb-9120-daa030689b56 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.016802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 52/109] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:01.021008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.022571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.023605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.024060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.024534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.025593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.027255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.066604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 58/110] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:39:00 2022] POST /identity/v3/projects => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:01.067748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.165476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2712-c52c-4453-aad8-4c04e93ab93a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.179981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f8b-d650-41ed-b0b2-550b064eebfc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.202971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.204176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.204686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.205205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.217947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.218640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.218931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.219153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:01.219252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a09f3b63-4bde-4d74-b398-c87c5419e850 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:01.229551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:01.230489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:01.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:01.234102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 53/111] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:01.235807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4fe9-09ec-48d9-bf6a-2367dee93414 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.236337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4fe9-09ec-48d9-bf6a-2367dee93414 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.236823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4fe9-09ec-48d9-bf6a-2367dee93414 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.242736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 54/112] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:01.244139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a4fe9-09ec-48d9-bf6a-2367dee93414 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.296870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.299085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bdded-e1fc-4052-9e6c-e686e5281dcf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.366808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a4fe9-09ec-48d9-bf6a-2367dee93414 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.401785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.402796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.403169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.403479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.414494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:01.422930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:01.423844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:01.424817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 55/113] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:01.426392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.426886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 59/114] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:01.428448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.429231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.429741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.450580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:01.452499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:01.531627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 60/115] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:01.532591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.536280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d30f-8c6e-4331-855a-75f2f200fd63 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.571135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.571824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.572211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.573536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.574321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:01.574741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.576274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.586162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 56/116] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:01.586998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.642828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c738c3-ea0d-4b91-819f-b5a862c005de None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.678754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.679425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.679709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.679984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.680866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:01.681222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.683100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.691651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 61/117] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:01.692316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.700237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78bef1-5ff4-4414-b45a-28cfb86cec97 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.735481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.736390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.736671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.736949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.738150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.739628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.741674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 57/118] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:01.742450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.793055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205964e5-3cc6-45b4-8a36-a9d73b216309 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.827299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.828237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.828543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.828826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.838920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:01.845538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621831-d139-4b0f-8cb3-e7e9f1958629 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:01.846651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:01.847482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:01.848232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 62/119] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:01.848895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.886658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.887557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.887859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.888123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.889368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.890854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.892802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 58/120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:01.893481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:01.955252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabef91-58ed-4b9c-80a5-dd7fab4f6eb7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:01.986415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:01.987305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.987586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.987866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.989092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:01.990533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6d74a-9b6e-43d3-aead-bff7191cf780 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:01.992344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 63/121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:01.993368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4f15-55a6-46b2-8b73-0b63c25ec056 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:01.993657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4f15-55a6-46b2-8b73-0b63c25ec056 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:01.994383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4f15-55a6-46b2-8b73-0b63c25ec056 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:01.995721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 64/122] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:01.996553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a4f15-55a6-46b2-8b73-0b63c25ec056 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.000842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662bc1b-2527-4293-b284-a103b3fd3154 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.035320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.036030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.036351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.036702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.037547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=e8639cfb66ff484fa419049d0399c82c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:02.037914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.039446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.052363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 59/123] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/role_assignments?user.id=e8639cfb66ff484fa419049d0399c82c&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=5189f106c6264cbf8e62606722de18a7 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.052363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.098982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a4f15-55a6-46b2-8b73-0b63c25ec056 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.134795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.135611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.135959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.136328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.137156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:02.137529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.139011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.154133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 65/124] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.155228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.182114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aafe7-4aa5-427e-8282-4dd83a30efbf None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.221122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.222146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.222507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.222858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.224181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.225780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabdba-3a33-4fcb-abf5-304d04e7c86c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.228064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 60/125] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.229931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.230259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.230602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.248502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:02.249841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:02.265683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3a189-9a50-4c7d-84c4-4c072c363bfc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.309495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.310660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.311004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.311393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.313258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.315291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.317722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 66/126] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.318467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.332087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 61/127] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:02.341168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.435319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235037f-5942-48ed-8943-3bd0a1d83431 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.449472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d46353-3276-4fa0-8a08-6181a3b80964 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.476125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.477589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.478132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.479165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.480865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.482465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.483584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.484555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 67/128] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.485682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.488475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.488919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.489700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/users/e279c42792694abda4ec83a94b9d0a59/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.491928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, user_id=e279c42792694abda4ec83a94b9d0a59, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.493606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.508589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 62/129] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:02 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/users/e279c42792694abda4ec83a94b9d0a59/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:02.509987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.621856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9029eeb-6ce3-4b3d-8b19-9e055fffd2d4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.659910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.661205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.661732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.662239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.674924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:02.683617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:02.684604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:02.685576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:02.687791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 63/130] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:02.688999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.696055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312108c5-bbb8-4967-a207-7d0d0da96104 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.761991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.762906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.763196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.763454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/users/5deadb4ca8544f08a5e703d8a595cc62/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.770192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, user_id=5deadb4ca8544f08a5e703d8a595cc62, role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.771716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.794684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 68/131] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:02 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/users/5deadb4ca8544f08a5e703d8a595cc62/roles/041b2fa99a644a7aba7f82618674fc9d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:02.795550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.807086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99ed-ee05-4dd2-8174-25f6166cbb99 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.844435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:02.845596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:02.846013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:02.846503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:02.847982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:02.849628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:02.851733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 64/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:02.859111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:02.960384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff986d63-bf66-4eda-b824-ca29ff0ee59c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:02.986900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389fc9-e34d-43db-9479-a94085a6912a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:03.021109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:03.021849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:03.022096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:03.022367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:03.023139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:03.023460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:03.024883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:03.025082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:03.027056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:03.027500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:03.027919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/users/5deadb4ca8544f08a5e703d8a595cc62/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:03.030323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, user_id=5deadb4ca8544f08a5e703d8a595cc62, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:03.031980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:03.034149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 69/133] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:03.039655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:03.050510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 65/134] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:02 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/users/5deadb4ca8544f08a5e703d8a595cc62/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:03.185033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cc78d-9b58-4222-935c-c86cbce571e1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:03.220159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:03.221027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:03.221300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:03.221583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:03.222822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:03.224338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:03.226323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 70/135] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:03 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:03.231806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:03.350780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b90e2-6d48-4387-9dd3-dc171f3aac87 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:03.397575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:03.398739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:03.399165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:03.399580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] PATH_INFO: `/v3/projects/0b284590702d411ebd823f81b2ce993f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:03.401677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=0b284590702d411ebd823f81b2ce993f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:03.403315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d21bc-3ceb-4516-b63d-6b11d23d7eb2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:03.405547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 66/136] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:03 2022] GET /identity/v3/projects/0b284590702d411ebd823f81b2ce993f => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:03.412977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:03.525298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d40e0-d8b2-4b20-bee0-ee0fc900c449 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:03.563655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:03.564597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:03.564884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:03.565172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] PATH_INFO: `/v3/projects/0b284590702d411ebd823f81b2ce993f/users/5deadb4ca8544f08a5e703d8a595cc62/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:03.567154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b284590702d411ebd823f81b2ce993f, user_id=5deadb4ca8544f08a5e703d8a595cc62, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:03.568724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b2bfe-457c-4d76-94f1-373fda21a566 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:03.591601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 71/137] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:03 2022] PUT /identity/v3/projects/0b284590702d411ebd823f81b2ce993f/users/5deadb4ca8544f08a5e703d8a595cc62/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:04.043002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b921d7-fdb6-4848-b581-bb1955c3d67d None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:04.043373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b921d7-fdb6-4848-b581-bb1955c3d67d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:04.043562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b921d7-fdb6-4848-b581-bb1955c3d67d None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:04.045138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 67/138] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:04.050883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:04.051177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:04.051462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:04.073306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:04.074588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:04.162687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 72/139] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:04.203376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b921d7-fdb6-4848-b581-bb1955c3d67d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:04.351203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b921d7-fdb6-4848-b581-bb1955c3d67d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:04.393525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:04.394343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:04.394624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:04.394899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:04.396097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:04.397591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a0b9b-a84f-44a0-808e-50581d30ad26 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:04.399277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 68/140] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:04.405227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:04.528060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00dc8e-0210-4a9b-b55b-59bbcf1577f1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:04.571088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:04.571785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:04.572142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:04.572449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:04.573198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:04.574654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf310873-4d51-4a7d-bb33-8dfe3f217e31 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:04.605924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 73/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:04 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:05.361528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cee1-6c05-4ee3-b971-ea0dbc3a5ecc None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:05.362033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cee1-6c05-4ee3-b971-ea0dbc3a5ecc None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:05.362128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311cee1-6c05-4ee3-b971-ea0dbc3a5ecc None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:05.363445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 69/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:05.369183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:05.370040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:05.370040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:05.387157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:05.388387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:05.464448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 74/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:05.491277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311cee1-6c05-4ee3-b971-ea0dbc3a5ecc None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:05.617335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311cee1-6c05-4ee3-b971-ea0dbc3a5ecc None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:05.662538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:05.663354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:05.663636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:05.663913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:05.678501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:05.686317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:05.687167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:05.687921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 70/144] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:05.693571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:05.809791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b49031-b152-441c-8e46-f2f65e69b6dd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:05.847914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:05.848627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:05.848910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:05.849187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:05.849949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] RBAC: Adding query filter params (name=reader) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:05.850254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:05.851650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:05.861417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 75/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:05.867256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:05.970358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cafec-2af5-4cd6-9eef-73023bd7eed1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.008853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.009849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.010341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.010779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] PATH_INFO: `/v3/users/e8639cfb66ff484fa419049d0399c82c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.012041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] RBAC: Authorizing `identity:get_user(user_id=e8639cfb66ff484fa419049d0399c82c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:06.013543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0a2e7-c118-4814-a2e1-6c63901e6a04 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:06.015199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 71/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:05 2022] GET /identity/v3/users/e8639cfb66ff484fa419049d0399c82c => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:06.021116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.114780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c2edb-765d-4397-a912-94402091eaca None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.115431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c2edb-765d-4397-a912-94402091eaca None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.115716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c2edb-765d-4397-a912-94402091eaca None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.117629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 72/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:06.132341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aab2a-80a9-462d-9e8d-9a8f4d9e1a78 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.132757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aab2a-80a9-462d-9e8d-9a8f4d9e1a78 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.133068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aab2a-80a9-462d-9e8d-9a8f4d9e1a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.150274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281ecd-fb46-493f-8ad4-5be70e3ce67a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.166840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3e1aab2a-80a9-462d-9e8d-9a8f4d9e1a78 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:06.168743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3e1aab2a-80a9-462d-9e8d-9a8f4d9e1a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:06.210198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.211083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.211370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.211649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.213053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:06.215831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:06.218682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 76/148] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:06.224474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.310346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 73/149] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:06.317966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a4e41-e622-4079-8c45-ae9fac70c127 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.318257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a4e41-e622-4079-8c45-ae9fac70c127 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.318536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a4e41-e622-4079-8c45-ae9fac70c127 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.319899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 74/150] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:06.326055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.326344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.326626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.354415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:06.355643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:06.405151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848503-070e-46b9-9a3c-98abd9d146bf None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.446616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.447588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.447879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.448166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/users/e8639cfb66ff484fa419049d0399c82c/roles/5189f106c6264cbf8e62606722de18a7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.450212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, user_id=e8639cfb66ff484fa419049d0399c82c, role_id=5189f106c6264cbf8e62606722de18a7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:06.451698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:06.462776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 75/151] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:06.463673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.476227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 77/152] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:06 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/users/e8639cfb66ff484fa419049d0399c82c/roles/5189f106c6264cbf8e62606722de18a7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:06.503466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.577679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12feaa-e1f4-4c24-85e7-5a7bd1c4386e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.625758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.626594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.626875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.627148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.649503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:06.654040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88020c56-f9ab-4c63-a731-c2000f31660e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:06.665474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:06.667701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:06.669667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 76/153] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:06.676319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.698201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.699053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.699343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.699621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.715642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:06.724060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:06.726132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:06.727949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:06.733795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 78/154] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:06.739773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.806657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f34096-3b2e-4865-ac37-a4e8e7749c9f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.846918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.847608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.847902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.848175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.848977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:06.849274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:06.850698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:06.864136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 77/155] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:06.870093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.870401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96ffd-27a0-461b-b2a5-ea8bee4bf334 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:06.911829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:06.912548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.912838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.913123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.913899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:06.914214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:06.915822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7069-bf23-4f34-ab52-2646ea768f2f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:06.933396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 79/156] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:06.934972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231dc4a-0650-4422-bbab-768fb5b8deb7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:06.935270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231dc4a-0650-4422-bbab-768fb5b8deb7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:06.935635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231dc4a-0650-4422-bbab-768fb5b8deb7 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:06.937011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 80/157] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:06.939441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231dc4a-0650-4422-bbab-768fb5b8deb7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:06.981023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3b1a-8787-46a5-b129-6f36a315e889 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.023503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.024456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.024734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.025085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] PATH_INFO: `/v3/users/e279c42792694abda4ec83a94b9d0a59` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.028987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] RBAC: Authorizing `identity:get_user(user_id=e279c42792694abda4ec83a94b9d0a59)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:07.030492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82a807-d510-4c7c-ac0a-2065bcc3aebf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:07.032545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 78/158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/users/e279c42792694abda4ec83a94b9d0a59 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:07.033796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.034077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.034354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.065425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:07.066631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:07.077507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231dc4a-0650-4422-bbab-768fb5b8deb7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.127637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.128573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.128965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.129690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.131398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:07.133216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:07.139618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 81/159] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:06 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:07.140576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:07.190448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 79/160] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:07.191265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:07.264568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72946c-802b-490e-9921-c64ffe7bcd0d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.317953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.318626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.318917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.319193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.344478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a14620-cf14-4903-97da-4072291a26d6 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.393540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.394207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.394489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.394763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.410430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:07.412656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:07.514554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 82/161] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:07.515363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:07.561466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:07.563641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:07.652265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93abca4-465e-4316-ad19-be9b4b147f09 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.712410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 80/162] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:07.713331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:07.753002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.753831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.754114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.754744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.785286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-4f5d7947-cb34-4e8d-8442-08b02fe662d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:07.801438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:07.802446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:07.803218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 83/163] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:07.804340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a130a5-cb4f-4a88-b53a-35e5ab03f011 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.804340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a130a5-cb4f-4a88-b53a-35e5ab03f011 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.804627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a130a5-cb4f-4a88-b53a-35e5ab03f011 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.806014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 84/164] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:07.806893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a130a5-cb4f-4a88-b53a-35e5ab03f011 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:07.818108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace159e0-8e72-477c-80ab-f08039655b4b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:07.865545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:07.866847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:07.867770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:07.873775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:07.875329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:07.877692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:07.881938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 81/165] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:07.883522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.061707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a130a5-cb4f-4a88-b53a-35e5ab03f011 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.108477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d7c1a-545a-4df6-818e-dc8805eb6e70 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.130602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.131510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.131797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.132117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.135320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:08.136849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d2a09-82c2-403b-a899-cfb36d44aeb3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:08.138612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 85/166] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:08.144823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.144823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.144823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.180881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.181673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.181957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.182233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.182995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:08.188398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:08.189305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:08.189873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:08.190489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:08.208024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 82/167] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:08.209057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.320950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 86/168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:08.322376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.427905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f878-2646-4e67-93de-590dff391ead None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.507825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fca43-5cf6-4979-867e-bcb3a62e108b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.509963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.510646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.510928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.511207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.512057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=e279c42792694abda4ec83a94b9d0a59) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:08.512415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:08.513894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:08.529234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 83/169] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/role_assignments?user.id=e279c42792694abda4ec83a94b9d0a59&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:08.531763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.581001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.581611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.581896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.582174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.583045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:08.583354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:08.584918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:08.599788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 87/170] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:08.601160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.773148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ebeaa-1d44-4f3a-af8b-15ad18cbff5e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.828187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c243790-e8d6-4c55-b003-74741b6a9ade None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:08.862592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.863514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.863801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.864078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.869212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:08.870743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036b5d8-136c-4893-b5d2-6caa62e4f9e3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:08.872716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 84/171] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:08.873858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea14e7-58b1-4638-be1a-a226338e4636 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.874119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea14e7-58b1-4638-be1a-a226338e4636 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.874386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea14e7-58b1-4638-be1a-a226338e4636 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.875802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 85/172] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:08.877489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:08.878427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.878737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.878967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.880175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea14e7-58b1-4638-be1a-a226338e4636 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:08.892366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-1befce44-b061-43f4-b860-451fc51c9157 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:08.901471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:08.902395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:08.904096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 88/173] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:08.904775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:08.905106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:08.905250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:08.924379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:08.925589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:08.999550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 89/174] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:09.000405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.110646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c8e6-1764-4cee-8fab-1776a72503aa None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.133078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea14e7-58b1-4638-be1a-a226338e4636 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.156685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.157382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.157666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.157939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.158722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:09.159034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.160516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.171427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 90/175] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:09.172472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.215987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.216699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.217451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.217735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.294893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.296422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.362526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65765006-db5a-4bfd-8c80-6ac044d4f815 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.397313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 86/176] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:09.398183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.421640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.422464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.422747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.423024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.445793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:09.453804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:09.455802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 91/177] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:09.457708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.498322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd37af-f10c-4c9c-bbdc-b6d4616ea332 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.532445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.533336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.533616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.533886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] PATH_INFO: `/v3/users/5deadb4ca8544f08a5e703d8a595cc62` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.535104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] RBAC: Authorizing `identity:get_user(user_id=5deadb4ca8544f08a5e703d8a595cc62)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.536596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.539415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 87/178] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/users/5deadb4ca8544f08a5e703d8a595cc62 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:09.539415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.636571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ee48-95f6-4f27-a88b-c1d9abfaa73b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.640129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d9daf-1fcc-452f-95f4-499bc4bb0103 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.676163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.676898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.677187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.677562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.678536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:09.678852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.680306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.686143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.687041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.687317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.687585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.690714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 88/179] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:09.691677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.691778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.693279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.695145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 92/180] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:09.700705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:09.794411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082939d1-15bd-4952-8287-98a411af5910 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.828045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.828742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.829027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.829308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.860081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832da0-5447-404e-bc87-e315de44d093 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:09.897511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.898959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1aa56-2112-49ae-8a7f-f501d1977985 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.902209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:09.902453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:09.902738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:09.903013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:09.915945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:09.917389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:09.919289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 93/181] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:09.919877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.000290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 89/182] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:10.005773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7a62-8d50-43c7-b4d3-8bb2d32e1705 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.006059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7a62-8d50-43c7-b4d3-8bb2d32e1705 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.006334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7a62-8d50-43c7-b4d3-8bb2d32e1705 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.007809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 90/183] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:10.008708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbebe-a46f-4fe6-9bb5-41cc8e73aad2 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.009063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbebe-a46f-4fe6-9bb5-41cc8e73aad2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.009519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbebe-a46f-4fe6-9bb5-41cc8e73aad2 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.010885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 91/184] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:10.015361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbebe-a46f-4fe6-9bb5-41cc8e73aad2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.030543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2c195-f004-408c-a84f-a9e36508971d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.065192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.065861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.066146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.066417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.067255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:10.067567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.071269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.084356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 94/185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:09 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.084356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.138077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbebe-a46f-4fe6-9bb5-41cc8e73aad2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.183954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.184679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.184965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.185409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.202761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b9b3-49ef-430d-916b-a333dd0c2f32 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.238647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.239639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.239991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.240380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] PATH_INFO: `/v3/projects/0b284590702d411ebd823f81b2ce993f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.241768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] RBAC: Authorizing `identity:get_project(project_id=0b284590702d411ebd823f81b2ce993f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.243335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04621a-8cfc-4fad-9d6d-3d981f52e357 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.246188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 95/186] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/projects/0b284590702d411ebd823f81b2ce993f => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.247814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9ffd-370c-48ab-bdbc-4add7f1f811d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.248169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9ffd-370c-48ab-bdbc-4add7f1f811d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.248666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b9ffd-370c-48ab-bdbc-4add7f1f811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.267410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cc7b9ffd-370c-48ab-bdbc-4add7f1f811d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:10.268704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cc7b9ffd-370c-48ab-bdbc-4add7f1f811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:10.270047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.272420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.351519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 96/187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:10.353125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.353638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.353967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.376491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:10.378680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:10.409631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 92/188] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:10.411784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.458581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 97/189] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:10.465278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.511560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69d44b-9de7-4b0d-9b3e-508ea128125f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.547659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.548283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.548552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.548831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.549664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=0b284590702d411ebd823f81b2ce993f, user.id=5deadb4ca8544f08a5e703d8a595cc62) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:10.549978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.551433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.557416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 93/190] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/role_assignments?user.id=5deadb4ca8544f08a5e703d8a595cc62&scope.project.id=0b284590702d411ebd823f81b2ce993f&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.558217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.570177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab8d12-a6ec-4bcb-aef2-a8469dbb4e1c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.605813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.606644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.606915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.607180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.619316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:10.628438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:10.629913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:10.630840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:10.632975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 98/191] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:10.633990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.662274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433a13-53cf-4634-bcaf-3dceae4e5a4c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.697902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.698792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.699073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.699348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.700669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.702166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b2a58-525e-45c7-821c-8c69689d44eb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.708849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 94/192] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.710621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e457-255e-4674-9cc6-3b6d132e7974 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.710912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e457-255e-4674-9cc6-3b6d132e7974 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.711197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e457-255e-4674-9cc6-3b6d132e7974 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.712745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 95/193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:10.713654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e457-255e-4674-9cc6-3b6d132e7974 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.756613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb8c58-5744-4609-b821-66c50f88afad None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.794409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.795180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.795841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.795841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.796944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.798475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.800210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 99/194] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.800968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.833013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e457-255e-4674-9cc6-3b6d132e7974 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.864727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.865391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.865668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.865939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.866800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:10.867114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.868518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858d92-2a1b-4bf4-8ab4-bd62eecb74f8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.876324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 96/195] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.881383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.881383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.881383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.896174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:10.897330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:10.897558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1dba1-3534-458b-9b43-a2209f557e33 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:10.930344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:10.930999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:10.931320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:10.931587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:10.932449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=7d83c4338860444eaf935fae1063be0c) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:10.932754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:10.934195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:10.939558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 100/196] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/role_assignments?user.id=7d83c4338860444eaf935fae1063be0c&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:10.940332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:10.978619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 97/197] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:10.982634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.052755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807e807-f2d4-44e7-9083-485f9643722c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.091087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.091876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.092161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.092477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.093193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.094674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.099026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8e1e-d4f7-4ea1-8054-9fcaf1386202 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.124027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 101/198] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:10 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:11.130029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.153290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.154209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.154493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.154771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.156114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.157677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.159706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 98/199] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:10 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:11.165397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.263046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366ed48-0a37-4d4a-88b5-c1ebd87ff03a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.302192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601c89e-47fd-47e4-91a1-648c4bcd197a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.302573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.303748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.304270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.304747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.318878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:11.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:11.328398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:11.329311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:11.331441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 102/200] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:11.338079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.351028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.351843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.352218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.352633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.440747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.442950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.462377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fa97e-a69e-4610-b553-dab68f122f76 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.500525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.501378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.501732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.502015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.502780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:11.503092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.505363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.514159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 103/201] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:11.519847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.566528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 99/202] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:11.572031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.627646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edb2dc-61d5-4e43-90b9-a2d2a0883f63 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.660683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.661989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.662554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.662841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] PATH_INFO: `/v3/users/e8639cfb66ff484fa419049d0399c82c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.664234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] RBAC: Authorizing `identity:get_user(user_id=e8639cfb66ff484fa419049d0399c82c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.665798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.667590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 104/203] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/users/e8639cfb66ff484fa419049d0399c82c => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:11.675930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.729709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeef99-a544-46a6-87e7-e75a7c5cded4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.786069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b0e67-37a5-4b5e-b0bf-26638d4101a0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:11.793939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.794828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.795109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.795387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.801184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.802528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:11.804299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 100/204] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:11.810209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:11.822465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:11.823463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:11.823949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:11.824447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:11.895609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:11.897928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:12.003428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 105/205] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:12.013245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:12.050494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6175c8-2490-49c0-a04c-527a4a74d073 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:12.112901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:12.113566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:12.113852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:12.114143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:12.115809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=7d83c4338860444eaf935fae1063be0c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:12.116130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:12.118302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:12.130769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 101/206] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:11 2022] GET /identity/v3/role_assignments?user.id=7d83c4338860444eaf935fae1063be0c&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:12.175895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87f43-56b8-4933-b850-fae04c36f907 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:12.213351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:12.214645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:12.215123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:12.215587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:12.217161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:12.218901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804afc1-fc5a-4b6e-b1cc-0874c189a4cc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:12.220948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:12 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:12.228338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:12.355571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d9c1d-664d-4f7f-9203-dcee8edac87e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:12.395949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:12.396585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:12.396871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:12.397147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:12.398025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, scope.project.id=77868d2f7582430694fddb0422e13efd, user.id=e8639cfb66ff484fa419049d0399c82c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:12.398410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:12.400048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e965f-7283-4de5-b49f-f3cdc78ec20d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:12.406871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:12 2022] GET /identity/v3/role_assignments?user.id=e8639cfb66ff484fa419049d0399c82c&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=5189f106c6264cbf8e62606722de18a7 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:13.402990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02675-7e54-4c7f-8f2e-268b708f90c6 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.403485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02675-7e54-4c7f-8f2e-268b708f90c6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.403485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02675-7e54-4c7f-8f2e-268b708f90c6 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.405116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:13.410617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.411008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.411170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.428145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:13.429374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:13.495745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 103/210] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:13.518662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02675-7e54-4c7f-8f2e-268b708f90c6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:13.631117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02675-7e54-4c7f-8f2e-268b708f90c6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:13.667897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:13.669085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.669557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.670028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.680527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-dc1b0d26-3f7b-4b4c-92dc-f7b0c2d3e7b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:13.688605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:13.689545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:13.690316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:13.692652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 108/211] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:13.701615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:13.800499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d7cc-e9b7-4896-8383-654fb6b20bbf None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:13.806047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884457-2c54-4652-936f-55ad07cf5931 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.806622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884457-2c54-4652-936f-55ad07cf5931 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.807120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44884457-2c54-4652-936f-55ad07cf5931 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.808759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 109/212] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:13.809675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6efb-6579-455f-9253-be84895e9410 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.809953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6efb-6579-455f-9253-be84895e9410 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.810511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6efb-6579-455f-9253-be84895e9410 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.811509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 110/213] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:13.816488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.816693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.817184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.833905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:13.834061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:13.834744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.835026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.835117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:13.835305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.836047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:13.836386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:13.837801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665bf4-b63d-41b5-a206-e0c9e8737507 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:13.846176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 104/214] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:13.849554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:13.849846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:13.850123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:13.866159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:13.867325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:13.903337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 111/215] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:13.909754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:13.940314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 105/216] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:13.941545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.011796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1302-7df5-41a7-8391-ae2aa76657f6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.038438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cbe0e-ad1b-4b96-91ed-fe373b509e55 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.050266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.051149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.051430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.051689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.053147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.054518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.060796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 112/217] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:13 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:14.060796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.072868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.074231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.074510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.074781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.075957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.077449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.083504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 106/218] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:14.083504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.159348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1280-3e5a-4be2-a0df-cd615a844ee7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.179355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb3bff-a955-4c27-b59f-a4433ea9635b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.193433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.194167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.194447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.194719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.204801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:14.212730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:14.213800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 113/219] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:14.219400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.220098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.220605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.220879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.221148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.222378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.223853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.225597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 107/220] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:14.226343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.325930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53ead-6455-4d98-aeca-c7bae7ff4589 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.326132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14eff-c098-4825-824b-551913d5dd92 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.358565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.359142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.359266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.359500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.359772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.359872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.360076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.360383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.360542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.361134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:14.361438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.361935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.362799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.370682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 108/221] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:14.376871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.385109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 114/222] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:14 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:14.386319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.494619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3aff-46d5-42b4-a97e-87cf28602552 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.496564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d4af9-701d-4964-95da-e3b44e9d7a64 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.535414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.535703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.536545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.536991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.537126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.537403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/7d83c4338860444eaf935fae1063be0c/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.537697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.538169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] PATH_INFO: `/v3/users/81c4cf44b7854738883da296a9a4f399` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.539713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, user_id=7d83c4338860444eaf935fae1063be0c, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.541998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c19f27-989c-45e0-86fc-bf9e866bb726 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.555838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] RBAC: Authorizing `identity:get_user(user_id=81c4cf44b7854738883da296a9a4f399)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.557956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.560452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 115/223] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/users/81c4cf44b7854738883da296a9a4f399 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:14.565914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 109/224] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:14 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/7d83c4338860444eaf935fae1063be0c/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:14.567360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.688875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05ac89-1755-495a-846d-51d5f72f994d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:14.716540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c456f-3c22-4fb0-8cd6-af252c367abd None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.717206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c456f-3c22-4fb0-8cd6-af252c367abd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.717680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c456f-3c22-4fb0-8cd6-af252c367abd None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.719577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 110/225] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:14.728780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.729214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.729623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.731530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:14.732205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:14.732515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:14.732790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:14.755440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:14.756612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:14.819899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:14.821492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:14.847145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 111/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:14.873365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:14.938477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 116/227] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:14.947958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.012904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b7f4-63aa-4303-a84d-a3d4736e282d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.046350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2e50f-c954-4abe-a687-f4e01d056372 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.047145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.047977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.048283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.048568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.059246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:15.067417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:15.068338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:15.069127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:15.076832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 112/228] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:15.082985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.088960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.090206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.090682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.091143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.092963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.094629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.096854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 117/229] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:14 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:15.105255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.186677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf18b40-373f-4ada-8270-bb63a962f453 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.220973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebcc2-0165-49a0-abab-3aea30393a1b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.228472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.228699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.229191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.229670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.230623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:15.231121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.232724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.241732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 113/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:15.247818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.264191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.265772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.266202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.266617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.267766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=81c4cf44b7854738883da296a9a4f399) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:15.268621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.270734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.278841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 118/231] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:15 2022] GET /identity/v3/role_assignments?user.id=81c4cf44b7854738883da296a9a4f399&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:15.371683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8361f5-58c7-493b-94a6-5e47427f97bc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.411840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.412797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.413076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.413347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.417124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.418632 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.420593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 114/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:15 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:15.426753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.573029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459531e5-799d-4494-9f05-ffc5635167cd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.606568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.607444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.607718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.607988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.609193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.610649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baaac3-c1d9-4bcd-bd5b-c31455664b75 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.612677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 119/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:15 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:15.620812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:15.751231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115fb51-bd3d-4ecc-a5d2-6636fa44c86f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:15.784452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:15.785403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:15.785690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:15.785970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/7d83c4338860444eaf935fae1063be0c/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:15.788213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, user_id=7d83c4338860444eaf935fae1063be0c, role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:15.789696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1cc27-9cd6-4117-bad6-d5c960ccf903 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:15.804929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 115/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:15 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/7d83c4338860444eaf935fae1063be0c/roles/041b2fa99a644a7aba7f82618674fc9d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:17.245948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800c76-d1ef-466c-ac14-7ad64e998537 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.246936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800c76-d1ef-466c-ac14-7ad64e998537 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.247487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b800c76-d1ef-466c-ac14-7ad64e998537 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.249139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 120/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:17.255048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.255838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.256356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.273633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:17.275239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:17.339509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354a16-7468-4256-babd-8b2b0eb35232 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.340126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354a16-7468-4256-babd-8b2b0eb35232 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.340612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f354a16-7468-4256-babd-8b2b0eb35232 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.342182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 121/236] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:17.346843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 116/237] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:17.355442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.355994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.356505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.374415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:17.375890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:17.393663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.503918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b77620-08bb-463d-bf8b-5a36cf88e896 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:17.526340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 122/238] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:17.548361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:17.549494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.549991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.550468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.565148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:17.573424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:17.574307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:17.575091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:17.577779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 117/239] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:17.583420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.584500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.683185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255fac-e87e-42bb-b2bf-f9a87cdb810a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:17.705894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261cc69-1101-4fea-a01b-f0f1ea82ebb6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:17.716954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:17.718090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.718562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.719033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.720438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:17.722100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca0152-6c5b-460b-9899-329789e95831 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:17.723914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 118/240] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:17.729838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b701d-be5e-4411-80c3-0df9a990f046 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.730375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b701d-be5e-4411-80c3-0df9a990f046 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.730841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b701d-be5e-4411-80c3-0df9a990f046 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.732662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 119/241] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:17.733531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b701d-be5e-4411-80c3-0df9a990f046 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.740533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:17.741308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.741588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.741861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.742609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:17.742907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:17.744479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614186-5f67-47da-825b-b30fccef0134 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:17.753539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 123/242] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:17.755277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.755449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.755674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.771918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:17.773113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:17.838259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 124/243] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:17.838942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.839735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b701d-be5e-4411-80c3-0df9a990f046 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:17.875573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:17.876686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.877182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.877642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.878548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:17.880579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:17.904486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 120/244] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:17 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:17.912650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:17.944468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989a406-7939-4fba-bc9a-938699c72461 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:17.981840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:17.982800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:17.983084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:17.983356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:17.984700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:17.986211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:17.988447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 125/245] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:17.998259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.039314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c9012-3f6b-402d-9851-6ec4630e8484 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.082912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.084136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.084636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.085118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.096184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:18.104634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:18.105587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:18.106357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:18.108855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 121/246] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:18.115130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.128571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11f90-6121-4de4-8914-27600a788962 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.210273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.210952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.211237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.211517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.268764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb78b-6c89-475a-a2ff-bc818f984101 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.297922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:18.300282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482b2c8-3517-47e5-8784-bd265b7edcaa None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:18.309044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.309743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.310025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.310303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.311075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:18.311382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:18.312884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462d61-2e60-4b70-9db8-0d9b4d77f421 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:18.322095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 122/247] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:18.323370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a3315-87b5-4dae-9a2c-40e0dd2da0f0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.323659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a3315-87b5-4dae-9a2c-40e0dd2da0f0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.323934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a3315-87b5-4dae-9a2c-40e0dd2da0f0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.325261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 123/248] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:18.327828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a3315-87b5-4dae-9a2c-40e0dd2da0f0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.414288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 126/249] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:18.415826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b7f82-34c8-4ae8-b5ac-151a93c858da None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.416106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b7f82-34c8-4ae8-b5ac-151a93c858da None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.416501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b7f82-34c8-4ae8-b5ac-151a93c858da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.434682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d72b7f82-34c8-4ae8-b5ac-151a93c858da None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:18.435935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d72b7f82-34c8-4ae8-b5ac-151a93c858da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:18.441309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a3315-87b5-4dae-9a2c-40e0dd2da0f0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.476711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.477604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.477926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.478243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] PATH_INFO: `/v3/users/de1bc182928c4e138f856216c549c5c1` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.492776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] RBAC: Authorizing `identity:get_user(user_id=de1bc182928c4e138f856216c549c5c1)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:18.494713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:18.496542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 124/250] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/users/de1bc182928c4e138f856216c549c5c1 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:18.497389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.519229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 127/251] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:18.520619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be4bb-1c20-4cc8-8f89-47a0e58282ee None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.520883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be4bb-1c20-4cc8-8f89-47a0e58282ee None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.521156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be4bb-1c20-4cc8-8f89-47a0e58282ee None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.522650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 128/252] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:18.523340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be4bb-1c20-4cc8-8f89-47a0e58282ee None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.614727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa392c-81c7-4dcf-abe1-e967a7fa9c0d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.648039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be4bb-1c20-4cc8-8f89-47a0e58282ee None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.650996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.652046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.652567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.653062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.654468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:18.656243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c052b-9d78-470a-9200-d0731511a5ee None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:18.658290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 125/253] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:18.659626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.660107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.660650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.677671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:18.679192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:18.685773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.686426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.686707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.686973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.752488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 126/254] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:18.753653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.767802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:18.769318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:18.867057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 129/255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:18.867898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.875003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf72c8-5432-4532-b403-a9ebd61cf00f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:18.917319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:18.918128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:18.918405 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:18.918675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:18.931739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:18.940131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:18.941108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:18.941907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:18.942529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 127/256] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:18.943040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:18.969357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530755-b2e7-42a3-b546-62c627881755 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.005034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.005892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.006266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.006635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.007568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=7d83c4338860444eaf935fae1063be0c) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:19.007976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.009688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.016164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 130/257] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/role_assignments?user.id=7d83c4338860444eaf935fae1063be0c&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.017020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.056263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a951ce-58d0-4613-97a4-bfa332f72e0d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.095008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.095978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.096396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.096819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.107464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:19.116076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:19.118032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 128/258] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:19.119988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.151284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98403b04-908b-4929-b30e-c3b8896d3a58 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.196188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.198035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.198617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.199865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.201302 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.202986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.205924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 131/259] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.207043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.265189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79050c-44d9-4837-9126-e14690d0779a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.345903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.346669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.346869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.347146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.347914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:19.348223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.349670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.367337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 129/260] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.368025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.383838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e12-2842-41ce-9c33-a466523aa068 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.429070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.429917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.430564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.430845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.432013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:19.432349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.433985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.447603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 132/261] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.448663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.490964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f479b1-e4d4-4147-9fbb-9f98debe1c5a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.526257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.526920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.527202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.527478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.528347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=de1bc182928c4e138f856216c549c5c1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:19.528659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.530131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.536429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 130/262] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/role_assignments?user.id=de1bc182928c4e138f856216c549c5c1&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=5189f106c6264cbf8e62606722de18a7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.537314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.560681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a23a0-4a3e-4b1e-bda6-12a4c2aef822 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.599271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.600684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.600981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.601525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] PATH_INFO: `/v3/users/81c4cf44b7854738883da296a9a4f399` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.603189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] RBAC: Authorizing `identity:get_user(user_id=81c4cf44b7854738883da296a9a4f399)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.605873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.607716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 133/263] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/users/81c4cf44b7854738883da296a9a4f399 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.614937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.660583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c4af1-3048-4947-9e78-5db9b5dffc66 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.695610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.697069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.697625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.698093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] PATH_INFO: `/v3/users/7d83c4338860444eaf935fae1063be0c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.699576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] RBAC: Authorizing `identity:get_user(user_id=7d83c4338860444eaf935fae1063be0c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.701268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.703147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 131/264] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/users/7d83c4338860444eaf935fae1063be0c => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:19.713547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.736566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266bf0-c424-4227-ab80-b798f45842ca None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.779329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.783064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.784368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.789185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.791053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.792808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.797144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 134/265] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:19.803587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:19.918870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6d87-d2e8-4b1b-bcb4-acf671e6d50c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.925418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8c8e4-11dc-4e4a-b4ce-0e820a5e27fb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:19.959991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.960911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.961186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.961461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/81c4cf44b7854738883da296a9a4f399/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.963500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, user_id=81c4cf44b7854738883da296a9a4f399, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:19.965083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:19.968400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:19.969672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:19.969925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:19.970176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:19.987078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 135/266] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:19 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/81c4cf44b7854738883da296a9a4f399/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:20.076060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:20.077620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:20.199517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 132/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:20.204781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:20.351655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cff29-6aa4-4700-bff7-03d1c4803331 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:20.386585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:20.387465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.387741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.388014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.389233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:20.390685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed01047-0b5a-4b1a-aaf4-4cdd3e1a0dd5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:20.392716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 136/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:20 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:20.398520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:20.515547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd9de-0805-44a2-94c4-5a62571606d2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:20.567083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:20.567750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.568024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.568319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.569282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=7d83c4338860444eaf935fae1063be0c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:20.569595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:20.571176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc944b8-8dcd-4940-a7d5-a4d3a3a29166 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:20.580178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 133/269] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:20 2022] GET /identity/v3/role_assignments?user.id=7d83c4338860444eaf935fae1063be0c&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:20.815428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43900-704b-497e-b159-811b1f1a20ac None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.815771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43900-704b-497e-b159-811b1f1a20ac None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.816185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43900-704b-497e-b159-811b1f1a20ac None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.818408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 137/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:20.824842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.825267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.825681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.855300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:20.856555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:20.869343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199abd2-77b8-4a39-bf65-5f016806e378 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.869967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199abd2-77b8-4a39-bf65-5f016806e378 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.870434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199abd2-77b8-4a39-bf65-5f016806e378 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.872048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 138/271] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:20.877878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:20.878460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:20.878925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:20.903447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:20.907119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:20.972155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 134/272] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:20.992289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 139/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:20.999959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.028048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.113083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3847bf-ad66-4981-9f84-c0e76755eee0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.160102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8bb36-1ff1-4f42-abfb-a2a17f4a0e0d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.176994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.177963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.178604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.178882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.192364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:21.199924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.200828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.201170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.201500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:21.201694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:21.202516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:21.203287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:21.203875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:21.203941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 135/274] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:21.208290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:21.208290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 140/275] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:21.210389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.211629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.312938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaef3-e014-4513-998a-6ddbbe992f24 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.336432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918b26-23fa-403e-b6cd-463bb6a17fcd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.347523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.348497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.348993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.349492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.350451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:21.352066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:21.374138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.374731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.375015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.375291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.376120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:21.376460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:21.377921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:21.378194 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 141/276] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:21 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:21.394525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 136/277] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:21.400000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.513055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faddd14-22bf-42dd-8a5b-92d3538a880d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.549350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.550636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.551154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.551617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] PATH_INFO: `/v3/users/3c5737ca909c4c25a19fdd124aa6e33b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.567455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] RBAC: Authorizing `identity:get_user(user_id=3c5737ca909c4c25a19fdd124aa6e33b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:21.569408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdccd9-e91b-47c9-bbad-58a56e43d1ba None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:21.571916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 142/278] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:21 2022] GET /identity/v3/users/3c5737ca909c4c25a19fdd124aa6e33b => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:21.584482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:21.687516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284941c8-c36f-456f-b0b1-eed394ee2d3f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:21.723339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:21.723999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:21.724298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:21.724575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:21.725419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, user.id=3c5737ca909c4c25a19fdd124aa6e33b, scope.system=all) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:21.725726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:21.727193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9477c-4d8e-4ead-955d-eea6d8e68527 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:21.732959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 137/279] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 12 00:39:21 2022] GET /identity/v3/role_assignments?user.id=3c5737ca909c4c25a19fdd124aa6e33b&scope.system=all&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:22.218131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933dc6-f949-4276-9b1a-9b8b1e6e80ff None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.218460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933dc6-f949-4276-9b1a-9b8b1e6e80ff None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.218685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933dc6-f949-4276-9b1a-9b8b1e6e80ff None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.220220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 143/280] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:22.225780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.226293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.226766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.244067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:22.245650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:22.316346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 138/281] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:22.335517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba933dc6-f949-4276-9b1a-9b8b1e6e80ff None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:22.471033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba933dc6-f949-4276-9b1a-9b8b1e6e80ff None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:22.517924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:22.518731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.519008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.519281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.537081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:22.545039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:22.545921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:22.547117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 144/282] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:22.553858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:22.703931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1520-b2c3-4162-b784-a86817bf99cc None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:22.758365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:22.759557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.760055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.760545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.761494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:22.761992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:22.763571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540707-1507-4958-9b48-51d3302342b5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:22.772053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 139/283] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:22.778166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:22.786971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5245-53f0-49af-b74d-071a5e7ec3e3 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.787525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5245-53f0-49af-b74d-071a5e7ec3e3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.787995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5245-53f0-49af-b74d-071a5e7ec3e3 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.789856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 140/284] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:22.794504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:22.795003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:22.795471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:22.817865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:22.820065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:22.906825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 141/285] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:22.949047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:22.994615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5a1e1-4e3d-4b2c-9490-e618aa48488d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.038724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.040045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.040597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.041112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] PATH_INFO: `/v3/users/de1bc182928c4e138f856216c549c5c1` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.042583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] RBAC: Authorizing `identity:get_user(user_id=de1bc182928c4e138f856216c549c5c1)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.044236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:23.046245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 145/286] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:22 2022] GET /identity/v3/users/de1bc182928c4e138f856216c549c5c1 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:23.052128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:23.138695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a6a61-14fa-4551-bbc8-2746ea88bb55 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.151399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9576fdc-a620-4441-a29a-9ee6c351bcc4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.185416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.185620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.186445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.186792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.186993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.187185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.187615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.188046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.188548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.190232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:23.192146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 146/287] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:23 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:23.199693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:23.205488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:23.221751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:23.222640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:23.223411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:23.225838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 142/288] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:23.232026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:23.309041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1f3-76a4-42bf-88af-81c4b7e45898 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.356966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcf6ec-9f09-4c5e-be7b-ab51552e1b27 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.370104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.371069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.371359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.371634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/de1bc182928c4e138f856216c549c5c1/roles/5189f106c6264cbf8e62606722de18a7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.373674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, user_id=de1bc182928c4e138f856216c549c5c1, role_id=5189f106c6264cbf8e62606722de18a7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.375227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:23.404450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 147/289] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:39:23 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/users/de1bc182928c4e138f856216c549c5c1/roles/5189f106c6264cbf8e62606722de18a7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:39:23.409206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.409928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.410213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.410492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.411288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:23.411596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.413064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:23.432434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 143/290] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:23.438289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:23.581150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad75ff-5341-4bb2-b27b-f767f9da48be None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.636326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.637253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.637539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.637815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] PATH_INFO: `/v3/users/81c4cf44b7854738883da296a9a4f399` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.639087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4cf44b7854738883da296a9a4f399)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.640571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:23.642249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 148/291] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:23 2022] GET /identity/v3/users/81c4cf44b7854738883da296a9a4f399 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:23.648609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:23.755911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4922-6231-43c9-a12b-155089efbb29 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:23.795523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:23.796182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:23.796497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:23.796776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:23.905126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:23.906672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:24.040836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 144/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:24.046455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.148754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cbc7a-f3ee-45c0-a990-7b384f12d7e3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:24.184498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:24.185316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.185601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.185876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.187197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:24.188738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca315d-dd7e-474c-a8dd-6032c027440c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:24.190883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 149/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:24.196905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.326825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567350f7-becf-46a8-8526-5c45180798ad None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:24.376964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:24.377631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.377915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.378189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.379398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=81c4cf44b7854738883da296a9a4f399) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:24.379710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:24.381204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6d51b-f3e7-47fc-a0b4-30df18e02e98 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:24.387736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 145/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/role_assignments?user.id=81c4cf44b7854738883da296a9a4f399&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:24.437516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9901fd-3393-4fdb-8570-35bc2f70a99f None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.437799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9901fd-3393-4fdb-8570-35bc2f70a99f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.437987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9901fd-3393-4fdb-8570-35bc2f70a99f None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.439643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 150/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:24.444953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.445286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.445524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.450042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17672-be84-4018-966d-78583eada52f None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.450333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17672-be84-4018-966d-78583eada52f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.450613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17672-be84-4018-966d-78583eada52f None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.452236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 151/296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:24.466743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:24.468354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:24.474953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.475234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.475512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.505385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:24.506624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:24.559542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 146/297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:24.592145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.607118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 152/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:24.631035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.745394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b0465-d016-4e7c-b401-3e5aa72ec54f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:24.756362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a72c1-7689-441f-b041-f06c600e9b21 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:24.788281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:24.789554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.789849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.790636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.792750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:24.794207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411eb6b3-8650-4b9c-b66a-8265a774440f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:24.795859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 153/299] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:24.797117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fb5b-e895-4217-9a1e-68314007c065 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.797395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fb5b-e895-4217-9a1e-68314007c065 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.797664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fb5b-e895-4217-9a1e-68314007c065 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.798930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 154/300] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:24.802265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8fb5b-e895-4217-9a1e-68314007c065 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.803286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:24.804383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.804878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.805228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.816642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-329cfbb8-becb-4e7a-8bb5-325d155a445d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:24.825241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:24.826125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:24.826890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 147/301] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:24.828634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.828987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.829330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.845882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:24.847178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:24.910126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8fb5b-e895-4217-9a1e-68314007c065 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:24.924327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 148/302] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:24.925705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:24.949209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:24.950482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:24.951379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:24.951931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] PATH_INFO: `/v3/groups` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:24.952946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:24.955079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:24.972299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 155/303] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 12 00:39:24 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:24.973094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.049762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4b4a-38a3-44e8-bee9-fb846b1e5cdc None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.086426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.087112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.087438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.088222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.088624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:25.088932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.090346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.100594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 149/304] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:25.106166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e4667-1ffb-42fa-943e-68d9f5c715cd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.106540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.148669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.148669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.148893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.149179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.166188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:25.174809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:25.175676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:25.176530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 156/305] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:25.182168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.229539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebfdb8-47a8-43e2-9f80-769b32491eba None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.272743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.273656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.273935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.274207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] PATH_INFO: `/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.287779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] RBAC: Authorizing `identity:get_user(user_id=50d5e5752f3c4cb4bb1ba770f2e27263)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.289639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.296359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 150/306] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:25 2022] GET /identity/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:25.297950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.300653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9b83-c5c5-44fc-a4cb-ff73f74f2335 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.339811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.340509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.340792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.341065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.341805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:25.342107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.343634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.352146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 157/307] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:25.357656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.403801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87f15-50c2-4635-9f43-d5959d3b9be0 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.440463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.441557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.442032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.442498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.443526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, user.id=50d5e5752f3c4cb4bb1ba770f2e27263, scope.system=all) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:25.444030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.445794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.459527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486373be-c660-4ab7-beda-9e2e10741fa3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.460017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 151/308] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 12 00:39:25 2022] GET /identity/v3/role_assignments?user.id=50d5e5752f3c4cb4bb1ba770f2e27263&scope.system=all&role.id=5189f106c6264cbf8e62606722de18a7 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:25.491012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.491919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.492198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.492501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] PATH_INFO: `/v3/users/3c5737ca909c4c25a19fdd124aa6e33b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.493894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] RBAC: Authorizing `identity:get_user(user_id=3c5737ca909c4c25a19fdd124aa6e33b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.495544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672870fb-3927-4a72-aa4e-2223e008ce5f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.497617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 158/309] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:25 2022] GET /identity/v3/users/3c5737ca909c4c25a19fdd124aa6e33b => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:25.503343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:25.614063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46da36-8f93-4032-9ede-50064a6415b5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:25.652135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:25.652636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:25.652900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:25.653172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] PATH_INFO: `/v3/system/users/3c5737ca909c4c25a19fdd124aa6e33b/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:25.654694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3c5737ca909c4c25a19fdd124aa6e33b, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:25.656093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51afa3-bacb-488b-b4b7-dc11b4bd855f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:25.667740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 152/310] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 12 00:39:25 2022] PUT /identity/v3/system/users/3c5737ca909c4c25a19fdd124aa6e33b/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:26.129681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36836d-884c-4f7d-9b08-ccebbc3a5556 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.130203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36836d-884c-4f7d-9b08-ccebbc3a5556 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.130203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36836d-884c-4f7d-9b08-ccebbc3a5556 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.132123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 159/311] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:26.138709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.139169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.139455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.157081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:26.158278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:26.229446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 153/312] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:26.252697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36836d-884c-4f7d-9b08-ccebbc3a5556 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:26.358726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36836d-884c-4f7d-9b08-ccebbc3a5556 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:26.392490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:26.393262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.393544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.393815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.403814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:26.411543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:26.413604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:26.415686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 160/313] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:26.419215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:26.518825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fccfaf-26c2-4c78-a139-1f1c6bf25a05 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:26.552465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:26.553542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.554013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.554529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.555466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:26.556013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:26.557727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:26.566552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 154/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:26.572465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:26.672724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdef316-ce9a-45c2-bca9-60100e36da94 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:26.706373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:26.707590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.708068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.708595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] PATH_INFO: `/v3/users/de1bc182928c4e138f856216c549c5c1` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.709967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] RBAC: Authorizing `identity:get_user(user_id=de1bc182928c4e138f856216c549c5c1)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:26.711579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:26.713851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 161/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:26 2022] GET /identity/v3/users/de1bc182928c4e138f856216c549c5c1 => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:26.719539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:26.821716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f15eb-14af-452c-a566-518b6bcb6f66 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:26.857514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:26.858541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:26.859013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:26.859591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:26.927024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:26.928850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:27.026677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 155/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:27.032559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:27.135770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012bdce-a191-48bc-b47e-c0450bafcb8e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:27.176579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:27.177604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.177828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.178105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.179358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:27.180917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc847-34e8-4e44-ab09-cc499d257208 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:27.183338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 162/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:27 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:27.189263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:27.290756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67adec-b492-49f1-9b5c-0270102476f1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:27.325747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:27.326796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.327271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.327735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.328815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c, user.id=de1bc182928c4e138f856216c549c5c1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:27.329316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:27.330944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbbd6-bfd2-40f4-b8a1-9bb0d0dc2c0a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:27.337942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 156/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:27 2022] GET /identity/v3/role_assignments?user.id=de1bc182928c4e138f856216c549c5c1&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=5189f106c6264cbf8e62606722de18a7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:27.374170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbae-6178-4573-b396-b47f46f3d69b None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.374507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbae-6178-4573-b396-b47f46f3d69b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.374746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbae-6178-4573-b396-b47f46f3d69b None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.376226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 163/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:27.382511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.383111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.383573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.401688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:27.403249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:27.476150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 157/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:27.499096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601fbae-6178-4573-b396-b47f46f3d69b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:27.618253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601fbae-6178-4573-b396-b47f46f3d69b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:27.652377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:27.653255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.653539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.653818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.654973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:27.656452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec2ad7-de9c-4bf7-883f-65a70cdd21b2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:27.658242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 164/321] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:27.663724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:27.709969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c530-3b27-4b43-94b7-f2f15b6b8c81 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.710255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c530-3b27-4b43-94b7-f2f15b6b8c81 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.710525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9c530-3b27-4b43-94b7-f2f15b6b8c81 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.711900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 165/322] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:27.717122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.717412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.717687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.734518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:27.735641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:27.763322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c98e-07c8-46e3-8b47-14117821f10c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:27.798793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:27.799506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.799799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.800066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] PATH_INFO: `/v3/groups` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.800813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:27.802325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379a724-b8e9-4f88-924c-bc043b1f4715 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:27.802535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 166/323] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:27.824427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 158/324] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 12 00:39:27 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:27.829444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:27.935489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a0dc3-e0dc-44bb-8ddf-49ac906d48bd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:27.971650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:27.975707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.975707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.975707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.975990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1d041-2dec-4db1-9e44-cee0e7528e40 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.975990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1d041-2dec-4db1-9e44-cee0e7528e40 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.975990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1d041-2dec-4db1-9e44-cee0e7528e40 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.975990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 159/325] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:27.982020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:27.982371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:27.982562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:27.985914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:27.994180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:27.995155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:27.995954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 167/326] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:28.001437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.001677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:28.002965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:28.084403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 160/327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:28.108196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.113912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75361-8893-4cd4-8e4d-eb8a265adc19 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.154543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99448003-678b-4094-9616-ae13dce74420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.155218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448003-678b-4094-9616-ae13dce74420 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.155493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448003-678b-4094-9616-ae13dce74420 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.155764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448003-678b-4094-9616-ae13dce74420 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.158987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99448003-678b-4094-9616-ae13dce74420 None admin] RBAC: Adding query filter params (name=reader) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:28.158987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99448003-678b-4094-9616-ae13dce74420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.159182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99448003-678b-4094-9616-ae13dce74420 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.168916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 168/328] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:28.174352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99448003-678b-4094-9616-ae13dce74420 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.236920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc9b9-cdfa-4697-8384-c02135e3941a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.284841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.286109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.286591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.287060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.294094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99448003-678b-4094-9616-ae13dce74420 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.302738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:28.310965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:28.311873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:28.312695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:28.315449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 161/329] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:28.325296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.333569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.334466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.334741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.335009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] PATH_INFO: `/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.336215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] RBAC: Authorizing `identity:get_user(user_id=50d5e5752f3c4cb4bb1ba770f2e27263)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.337861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.340787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 169/330] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:28.347758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.452800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879b3f-207f-43e4-bbfb-41cce414e141 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.463856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02f508-9880-4f55-97bb-90ed3d4984f2 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.493566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.494670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.495163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.495648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.496662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:28.497289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.499872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.509358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 162/331] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:28.516178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.522177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.523162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.523517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.523865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] PATH_INFO: `/v3/system/users/50d5e5752f3c4cb4bb1ba770f2e27263/roles/5189f106c6264cbf8e62606722de18a7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.525888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=50d5e5752f3c4cb4bb1ba770f2e27263, role_id=5189f106c6264cbf8e62606722de18a7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.527747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.543322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 170/332] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 12 00:39:28 2022] PUT /identity/v3/system/users/50d5e5752f3c4cb4bb1ba770f2e27263/roles/5189f106c6264cbf8e62606722de18a7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:39:28.638614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864d4f-dd38-4bdd-bf34-ad73807126e7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.688546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.690309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.691031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.691698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] PATH_INFO: `/v3/users/3c5737ca909c4c25a19fdd124aa6e33b` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.696866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] RBAC: Authorizing `identity:get_user(user_id=3c5737ca909c4c25a19fdd124aa6e33b)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.699134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f7501-ea50-4fe2-a356-a8c025c03b88 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.702075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 163/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/users/3c5737ca909c4c25a19fdd124aa6e33b => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:28.709443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:28.845635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49f5c-5704-4e11-8d8b-c7fec7af3771 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:28.885529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:28.886199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:28.886480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:28.886748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:28.887588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] RBAC: Adding query filter params (role.id=d2a59677d1e9405f8eeb3527dd367536, user.id=3c5737ca909c4c25a19fdd124aa6e33b, scope.system=all) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:28.887895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:28.889457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514508cc-7029-4b1a-9c7d-b3fa4ac1521b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:28.903835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 171/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 12 00:39:28 2022] GET /identity/v3/role_assignments?user.id=3c5737ca909c4c25a19fdd124aa6e33b&scope.system=all&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:30.930403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e1f5-2e88-42e7-81d4-b4d1b67c179f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:30.931331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e1f5-2e88-42e7-81d4-b4d1b67c179f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:30.931767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871e1f5-2e88-42e7-81d4-b4d1b67c179f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:30.933362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 164/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:30.939467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c646-d7e6-41d6-867d-62720a81dd57 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:30.940017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c646-d7e6-41d6-867d-62720a81dd57 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:30.940289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815c646-d7e6-41d6-867d-62720a81dd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:30.961671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8815c646-d7e6-41d6-867d-62720a81dd57 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:30.963167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8815c646-d7e6-41d6-867d-62720a81dd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:31.044699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 172/336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:31.080147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871e1f5-2e88-42e7-81d4-b4d1b67c179f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:31.231893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871e1f5-2e88-42e7-81d4-b4d1b67c179f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:31.245197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81313f-12ca-430f-8ca1-ca1dbe7243a2 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.245838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81313f-12ca-430f-8ca1-ca1dbe7243a2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.246313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81313f-12ca-430f-8ca1-ca1dbe7243a2 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.247905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 173/337] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:31.256181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.256780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.257265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.282351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:31.283364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:31.284435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.284918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.285399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.286783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:31.310054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:31.318477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:31.319380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:31.320176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:31.327109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 165/338] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:31.331397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:31.435062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 174/339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:31.455437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcb919-13a2-49f2-992d-ca78409b07f5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:31.474606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:31.496903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:31.497564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.497848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.498128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.498952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:31.499265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:31.501322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:31.517746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 166/340] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:31.524787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:31.679651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f18a16-6ff8-461a-bcf9-951ac5d1b9a1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:31.686332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e15f9-3d0b-4bda-a25d-bf043ca8ec36 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:31.718648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:31.719403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.719757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.720116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.727453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:31.728669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.729151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.729603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.742557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-192f346a-2804-435b-ad64-74a5945a3cb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:31.752973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:31.754226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:31.755017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:31.757807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 175/341] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:31.760223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54089f-2d16-40a5-8c95-c1a428694db0 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.760681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54089f-2d16-40a5-8c95-c1a428694db0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.761116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54089f-2d16-40a5-8c95-c1a428694db0 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.767772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 176/342] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:31.769781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e41f1-b323-49fd-bec2-7ce9f888c79c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.770245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e41f1-b323-49fd-bec2-7ce9f888c79c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.771042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e41f1-b323-49fd-bec2-7ce9f888c79c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.776319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 177/343] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:31.777808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e41f1-b323-49fd-bec2-7ce9f888c79c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:31.812155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:31.813973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793418-ed0a-41e3-8288-535e9bbf6101 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:31.887146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e41f1-b323-49fd-bec2-7ce9f888c79c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:31.919198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 167/344] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:31.921161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.921679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.922113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.928130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:31.929052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.930275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.930275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.930659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] RBAC: Adding query filter params (name=member) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:31.931311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:31.933288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb4ecd-62e9-4eec-a6d8-e5341e65ccac None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:31.942702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:31.944670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:31.947061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 178/345] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:31.950108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:31.950108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:31.950479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:31.974462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:31.974858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:32.033391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 168/346] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:32.034345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.069595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 179/347] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:32.070728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.180465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9fc1b-52e1-49bc-bd3d-fa1837705653 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.200188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45fcee-30a4-4390-b6a3-86e4a7e48050 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.223519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.224391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.224671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.224941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.226134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:32.227663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:32.230038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 169/348] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:32.230823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.236540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.237227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.237489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.237764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.333582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:32.334960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8b7c-f72a-4782-a48d-8b8008858c63 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:32.355194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada55f-3553-4a32-afec-15236fb9c5e1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.398091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.399073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.399361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.399674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.410337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:32.419446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:32.420428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 00:39:32.421220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:32.422311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 170/349] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:32.422906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.454798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 180/350] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:32.461009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c100-68ea-453b-b646-57c300affc41 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.461292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c100-68ea-453b-b646-57c300affc41 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.461568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c100-68ea-453b-b646-57c300affc41 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.463114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 181/351] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:32.463834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c100-68ea-453b-b646-57c300affc41 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.551822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6d2fb-5dc9-4b0a-bf58-a5929e4176b8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.585146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c100-68ea-453b-b646-57c300affc41 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.592742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.594379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.594900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.595727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.615554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:32.624306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:32.625182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:32.625953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:32.626563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.626563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.626563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.627297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.632283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 171/352] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:32.632283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.635726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:32.636477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:32.638229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 182/353] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:32.639007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.763305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b8c6-5dd7-4abc-90f0-9d7381679084 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.765688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c41d74-5873-4148-a2de-dd279017aa93 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.807072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.808281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.808733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.809063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.809549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.809650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.809875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.810143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.811118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] RBAC: Adding query filter params (name=reader) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:32.811342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:32.811825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:32.812829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9adf5e-2b82-4c62-8141-4953b359a3d6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:32.813722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:32.814821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 183/354] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:32.818244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.818244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.818244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:32.829066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 172/355] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:32.830280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.839280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:32.839280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:32.930583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 184/356] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:32.932389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:32.951784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d59c5-0695-4ec8-bcf4-9610bbfe716a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:32.998256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:32.998953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:32.999224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:32.999502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.001317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:33.001317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.002024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.011475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 173/357] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.012551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.061603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a8b5f-2bc1-4cad-a72c-c2c6ec9c9ef8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.107521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.108202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.108533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.108808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.109656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=77868d2f7582430694fddb0422e13efd) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:33.109969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.111422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.117053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 185/358] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:32 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.117861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.138618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca1dd8-4e2c-48ba-b596-2efff82202ad None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.189096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.189997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.190273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.190542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.191904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.193680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.195618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 174/359] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.197403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.266570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e9b0a-0c12-4de9-a373-e3339c7b0f96 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.317087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.317728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.317840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.318211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] PATH_INFO: `/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.319260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] RBAC: Authorizing `identity:get_user(user_id=50d5e5752f3c4cb4bb1ba770f2e27263)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.320785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.323019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 186/360] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/users/50d5e5752f3c4cb4bb1ba770f2e27263 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.323334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4e550-a050-4b3b-a361-22416d374bfb None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.323637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.362542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.364156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.365163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.365163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.377036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-99934da4-0a8d-4c18-89ed-49a6a0fd703c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:33.385995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:33.386905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:33.387719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 175/361] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:33.389225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3aa3-7507-4500-a174-88a5edf97225 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.389803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3aa3-7507-4500-a174-88a5edf97225 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.389803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3aa3-7507-4500-a174-88a5edf97225 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.391300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 176/362] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:33.392019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3aa3-7507-4500-a174-88a5edf97225 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.454820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846abd84-f8e9-4894-a78c-e33ae4894e1f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.501085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.501967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.502311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.502642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.511584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3aa3-7507-4500-a174-88a5edf97225 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.558509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.559487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.559767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.560045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.560996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=77868d2f7582430694fddb0422e13efd) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:33.561290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.562723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.568933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 177/363] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.569400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.584047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.585645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b78f6-841b-40d1-9b77-64050a70048d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.702560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33451d4b-cc9c-43b7-8366-e5117d9635eb None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.713640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 187/364] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:33.715037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.715285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.715561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.738823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:33.740090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:33.746377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.747427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.747923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.748919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.750016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] RBAC: Adding query filter params (role.id=5189f106c6264cbf8e62606722de18a7, user.id=50d5e5752f3c4cb4bb1ba770f2e27263, scope.system=all) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:33.750551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.753062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.769948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 178/365] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/role_assignments?user.id=50d5e5752f3c4cb4bb1ba770f2e27263&scope.system=all&role.id=5189f106c6264cbf8e62606722de18a7 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.773165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.870098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 188/366] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:33.870892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:33.925763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e81f4-b725-4fe6-9003-aae1ec0ecba9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:33.964073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:33.964860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:33.965123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:33.965391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:33.966154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:33.966458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:33.967888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:33.985028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 179/367] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:33.986843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.090586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3c801-d272-4c9e-aca1-6fb3b59c402c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.099706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08c65e-6903-458a-a8cf-56f7c6004da9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.135378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.136226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.136575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.136847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.153492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:34.162526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:34.163513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:34.164978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 180/368] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:34.165784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.165784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.165784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.165784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.167015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.168845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.169170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.170823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 189/369] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:33 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:34.174854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.289895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666942-525a-4e08-a7fd-43be404fd38c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.293351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283e23-8a57-4fb2-af50-b7c81c082a19 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.324378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.325570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.325964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.326242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.329717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.330498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.330874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.331243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.332094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:34.332572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.334033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.345023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 190/370] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:34.345989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.416540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.418714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.457541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e35fa-5151-4dba-91fa-862059e94178 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.508126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.509193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.509474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.509747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.512107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.513637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.515607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 191/371] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:34.516425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.546769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 181/372] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:34.547580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.638607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a9f33-74fe-40c9-9db4-81b8667406eb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.674927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf563d-4776-47e1-8d5c-618e6e852e46 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.678652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.679432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.679838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.680217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.713330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:34.714309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:34.714727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:34.715227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:34.716293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:34.716805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.718872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.730608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 182/373] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:34.731993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.760712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:34.762333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:34.876415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 192/374] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:34.882017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:34.950432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3382-4ed0-4a06-a765-6acda9381f76 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:34.995783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2cc9-ae9b-40cb-b693-57600af3c8d7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:35.037884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.038100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.039123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.039495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.039791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.039936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] PATH_INFO: `/v3/projects/137a089466c04c34b8c9ec48a39d84b0` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.040408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.040837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.044274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] RBAC: Authorizing `identity:get_project(project_id=137a089466c04c34b8c9ec48a39d84b0)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.044274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.045298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.046051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 193/375] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/projects/137a089466c04c34b8c9ec48a39d84b0 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:35.047249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.051076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:35.052363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 183/376] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:34 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:35.054109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:35.165823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbe4e1-e18a-432c-9622-d9dce919776b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:35.203245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.204298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.204698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.205074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] PATH_INFO: `/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.211209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] RBAC: Authorizing `identity:get_group(group_id=045c970ae3b74298a4f0cb4bd129bf0b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.213823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.215991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 194/377] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:35 2022] GET /identity/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:35.222748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:35.286407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16895a1-ecdd-4f7e-b9e8-5ab3429f62c9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:35.332672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837dbac-efc8-4072-a6ed-e50fad84edf5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:35.371003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.371234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.371654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.371865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.372028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.372197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.372399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.372549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.373191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] RBAC: Adding query filter params (group.id=045c970ae3b74298a4f0cb4bd129bf0b, role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=137a089466c04c34b8c9ec48a39d84b0) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:35.373483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.375086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333d870-9d8d-475f-bd12-1c59cb3f2acc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.377029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.378494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.381724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 184/378] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:35 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:35.382267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 195/379] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:35 2022] GET /identity/v3/role_assignments?group.id=045c970ae3b74298a4f0cb4bd129bf0b&scope.project.id=137a089466c04c34b8c9ec48a39d84b0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:35.386798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:35.538181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a5cf-e378-4b36-ab6a-233093bcea14 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:35.574515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:35.575440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.575890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.576331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.577378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:35.577828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:35.579409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f2e8-087c-449b-b167-7c0af4d66bfb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:35.589197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 185/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:35 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:35.706968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637b19-e614-4ce6-b80d-602a717e70c2 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.707624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637b19-e614-4ce6-b80d-602a717e70c2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.708103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637b19-e614-4ce6-b80d-602a717e70c2 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.710118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 196/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:35.717162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:35.717755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:35.718239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:35.736896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:35.738483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:35.823375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 186/382] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:35.859785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a637b19-e614-4ce6-b80d-602a717e70c2 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:35.995467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a637b19-e614-4ce6-b80d-602a717e70c2 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.031315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.032531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.033040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.033564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.045286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:36.053988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:36.054931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:36.055794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:36.058237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 197/383] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:36.064025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:36.182434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ce85-ef0e-4d71-a91b-6995ce67a4c2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.222304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.222985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.223267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.223546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.224330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:36.224642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:36.226161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:36.238434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 187/384] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:36.244698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:36.351916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9fa0-d98f-49b2-aa57-358646f181cf None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.386192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.387070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.387355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.387631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.388853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:36.390402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce19d6-8078-459b-9cbb-ebb93e5102c4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:36.392372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 198/385] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:36 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:36.397605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:36.494259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d4d1-5871-42ec-9227-2173c41a859c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.494590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d4d1-5871-42ec-9227-2173c41a859c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.494857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4d4d1-5871-42ec-9227-2173c41a859c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.500156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 199/386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:36.501852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.502137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.502414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.517310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17bd15-9972-4d34-b1ea-e0a8343a2be4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.532523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:36.534092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:36.588022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.589780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.590472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.591199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.594812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:36.597428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:36.601422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 188/387] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:36 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:36.609175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:36.698189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 200/388] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:36.727517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:36.752517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96c076-3387-4f28-b799-04c415227b68 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.798439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.799723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.800229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.800727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/groups/e6f9eb6a96dc4273b614371e26aff828/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.809079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, group_id=e6f9eb6a96dc4273b614371e26aff828, role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:36.810829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:36.836963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 189/389] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 12 00:39:36 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/groups/e6f9eb6a96dc4273b614371e26aff828/roles/041b2fa99a644a7aba7f82618674fc9d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:36.855189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46422525-f37f-4132-b91f-2c28625dde89 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:36.895579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:36.896479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:36.896854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:36.897134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:36.909840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:36.918619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:36.919657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:36.920973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 201/390] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:36.926336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.083370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459db9d5-bc34-4a93-8f90-d22d95ae4984 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:37.121176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:37.121870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.122160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.122429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.123195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:37.123504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:37.124984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:37.134250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 190/391] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:37.139529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.242148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f37d3-610d-4c75-a9f6-73bd1c8a67a7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:37.278117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:37.279021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.279371 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.279655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.281094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:37.282595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:37.284654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 202/392] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:37 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:37.290964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.396274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf67e5-af85-46ca-9733-eeb75fadb282 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:37.430410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:37.431346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.431630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.431909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.433143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:37.434611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b66a9b-9692-489f-8c9c-5c3d5094f645 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:37.436343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 191/393] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:37 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:37.442215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.546956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d5b6-7e4b-4836-a706-87b1fb84d3a9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:37.569844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25198b-4774-45f3-94ae-2c3babd562f2 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.570150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25198b-4774-45f3-94ae-2c3babd562f2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.570418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25198b-4774-45f3-94ae-2c3babd562f2 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.573241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 192/394] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:37.578993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.579890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.580665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.590804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:37.591702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.591984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.592264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd/groups/e6f9eb6a96dc4273b614371e26aff828/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.597532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] RBAC: Authorizing `identity:create_grant(project_id=77868d2f7582430694fddb0422e13efd, group_id=e6f9eb6a96dc4273b614371e26aff828, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:37.598966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:37.600708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:37.602340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:37.632874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 203/395] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 12 00:39:37 2022] PUT /identity/v3/projects/77868d2f7582430694fddb0422e13efd/groups/e6f9eb6a96dc4273b614371e26aff828/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:37.693616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 193/396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:37.723339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.838238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4d496-240e-4c70-8c6f-0e247e2e5c7e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:37.874171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:37.875347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.875915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.876422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] PATH_INFO: `/v3/roles/member` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.888668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a71f1f36-3401-423a-b6a6-7126e8a6554c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:37.897242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:37.898104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:37.898944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:37.901924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 204/397] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:37.908037 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:37.953719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306d90-8d01-4ea8-bbf3-512f0b4f1c05 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.954350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306d90-8d01-4ea8-bbf3-512f0b4f1c05 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.955152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306d90-8d01-4ea8-bbf3-512f0b4f1c05 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.957151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 205/398] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:37.966369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:37.966990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:37.967558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:37.998689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:38.000949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:38.050981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1100de-3be5-4bb4-a34c-561814af8bf6 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.076086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 206/399] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:38.088506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.089217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.089502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.089773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.090534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:38.090839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.092306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.101600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 194/400] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:38.104216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.107858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.211122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37b44b-2d34-4580-ad3b-40ca7f5c5700 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.237526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac9ad6-c673-48e2-aac9-134c1f2bea19 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.245260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.246068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.246344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.246614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.257398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:38.265119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:38.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:38.266802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 207/401] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:38.277031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.283334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.284475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.286514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.288980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.290499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.295789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.297811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 195/402] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:38.303243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.428564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90e81c-f1e8-4e1a-8369-6bb0f385fe43 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.469135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.470349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.472454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.472964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.475982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.481425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.483721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 196/403] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:38.489487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.499984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6969d-0af3-437d-9bb4-e4ea375bc8a5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.551470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.552478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.552999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.553451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.554456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:38.554955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.557035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.573216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 208/404] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:38.578436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.618704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76689c-8125-45ee-b0e9-29d2703a0d28 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.653511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.654429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.654715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.654994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/groups/e6f9eb6a96dc4273b614371e26aff828/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.657185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, group_id=e6f9eb6a96dc4273b614371e26aff828, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.658705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.687554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 197/405] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 12 00:39:38 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/groups/e6f9eb6a96dc4273b614371e26aff828/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:38.759343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefbe5-b4b8-4a3a-b82a-51e5a85c512f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.795893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.796860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.797142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.797418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] PATH_INFO: `/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.798651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] RBAC: Authorizing `identity:get_group(group_id=045c970ae3b74298a4f0cb4bd129bf0b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.800218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.802381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 209/406] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:38.809604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.921525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94129471-aaff-48b0-b8da-c56bfad54c06 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:38.956743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:38.957631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.957913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.958189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] PATH_INFO: `/v3/projects/137a089466c04c34b8c9ec48a39d84b0` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.959429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] RBAC: Authorizing `identity:get_project(project_id=137a089466c04c34b8c9ec48a39d84b0)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:38.960999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63c02a-c22b-473d-ac75-29b5b52088a0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:38.963317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 198/407] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:38 2022] GET /identity/v3/projects/137a089466c04c34b8c9ec48a39d84b0 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:38.968646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:38.989358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef8b3-3077-4d74-a1a5-bdd9ec3a67c9 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.989700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef8b3-3077-4d74-a1a5-bdd9ec3a67c9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.989916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef8b3-3077-4d74-a1a5-bdd9ec3a67c9 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:38.991413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 199/408] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:38.997147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:38.997474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:38.997693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.023011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:39.024117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:39.101804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3afaa-4811-4b21-b97b-26a237fa333c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:39.139068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:39.139962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.140241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.141388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] PATH_INFO: `/v3/projects/137a089466c04c34b8c9ec48a39d84b0/groups/045c970ae3b74298a4f0cb4bd129bf0b/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.146526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] RBAC: Authorizing `identity:create_grant(project_id=137a089466c04c34b8c9ec48a39d84b0, group_id=045c970ae3b74298a4f0cb4bd129bf0b, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:39.148052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:39.172991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 210/409] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 12 00:39:38 2022] PUT /identity/v3/projects/137a089466c04c34b8c9ec48a39d84b0/groups/045c970ae3b74298a4f0cb4bd129bf0b/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:39.175186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 200/410] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:39.213509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:39.323072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2acd0-fd13-424a-a08c-1edd2f1937fb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:39.366039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:39.366899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.367186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.367466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.379900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:39.392648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:39.394823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:39.396835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 211/411] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:39.401085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:39.531543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609efd0-7ac7-40e6-89a7-3743dfc6533d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:39.564353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:39.565026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.565303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.565575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.566341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:39.566628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:39.568002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:39.577892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 201/412] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:39.583074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:39.718328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc1dd-857c-470a-b7fd-eb6b6d50f4f3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:39.762319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:39.763195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.763475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.763748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.764961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:39.766407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adbd83-96d1-4ef0-8059-4b59303f7fbb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:39.768352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 212/413] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:39 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:39.773509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:39.874839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3c8d-5dd9-45ee-b2b8-97d75d8cbc22 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.875495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3c8d-5dd9-45ee-b2b8-97d75d8cbc22 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.875982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab3c8d-5dd9-45ee-b2b8-97d75d8cbc22 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.877815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 213/414] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:39.889741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.890311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.890774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.929855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82402e8-34e7-4b3f-a40b-92cb7449e7df None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:39.931473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:39.932900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:39.969375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:39.970256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:39.970536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:39.970811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:39.973000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:39.974462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:39.976163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 202/415] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:39 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:39.982175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.065364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 214/416] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:40.086187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.184301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1c26d-0cfa-4cc0-b8dd-adac42d6c825 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:40.185256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f16d9-0fd5-4fe8-8d62-b6172e5ac8bf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:40.225954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:40.227152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.227656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.228123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.240016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:40.240946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.241229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.241503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/groups/e6f9eb6a96dc4273b614371e26aff828/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.248280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3184670dd5ef4e518db1f41ed2dc877c, group_id=e6f9eb6a96dc4273b614371e26aff828, role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:40.249759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dab09-3dba-413f-9cf7-52fedb34b7d5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:40.254279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:40.263126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:40.265130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:40.265911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 215/417] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:40.271421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.287776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 203/418] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 12 00:39:39 2022] PUT /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c/groups/e6f9eb6a96dc4273b614371e26aff828/roles/041b2fa99a644a7aba7f82618674fc9d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:39:40.423932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc24a-0d43-494f-ad1f-7a6932671946 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:40.472577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:40.473260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.473598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.473872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.474631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:40.474936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:40.476325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:40.481976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503fcb-0d15-462f-a99d-c8320672d8fe None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.482636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503fcb-0d15-462f-a99d-c8320672d8fe None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.483085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503fcb-0d15-462f-a99d-c8320672d8fe None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.484944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 204/419] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:39:40.488304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 216/420] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:40.491360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.491784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.492287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.492752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.515120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:40.516605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:40.611514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a0d9c-217f-4ec1-bfca-6d28e1007b47 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:40.627771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 205/421] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:40.646672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:40.647329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:40.647611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:40.647883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:40.669262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.727256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:40.729340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:40.843293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 217/422] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 00:39:40.848612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:40.946611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4c2e-dc44-45f9-baa7-7e6f0d63695f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:40.976532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313b8b-204e-4b15-a54b-026a7c2764c5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.025936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.026740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.027019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.027291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.028530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.030182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.032911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 218/423] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:40 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:41.039568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.044579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.045401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.045673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.045948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.066819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:41.080015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:41.081039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:41.081817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 206/424] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:41.088753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.196757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203ec88-75d7-4844-9aa9-b8c4410e211c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.235216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.237571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.239255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.239745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.245800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.247538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.249817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 219/425] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:41.256380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.369191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f703e-f7f9-48c4-af11-3c3f9c068b8a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.378410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e054a-d794-42b1-9ca6-94e619dc4b72 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.414843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.415941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.416454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.416906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.417914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=77868d2f7582430694fddb0422e13efd) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:41.418404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.420039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.426888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 220/426] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:41.436019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.436772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.437054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.437331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.438110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:41.438416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.439866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8fc15-062a-493d-b669-5551a823bff8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.459005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 207/427] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:41.464498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.565004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4f54-f0db-43d4-b0c7-51fc97bec5ff None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.565349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4f54-f0db-43d4-b0c7-51fc97bec5ff None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.565559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4f54-f0db-43d4-b0c7-51fc97bec5ff None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.567215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 208/428] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:41.572505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.572781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.573055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.586143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278a250-228e-474a-b9e7-5cd11ca59f36 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.613458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:41.614662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:41.631944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.633380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.633995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.634458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.724454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.725936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.741569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 209/429] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:41.768055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.837626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 221/430] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:41.844772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.880483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23538e0f-c788-4695-ae6a-5fe1eb33e885 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.917679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.918882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.919398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.919874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.930946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:41.940874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:41.941745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 00:39:41.942534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:41.945378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 210/431] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:41.945501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2581f-9d47-413f-b1b1-3328940ba56c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:41.954092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:41.988955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:41.989883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:41.990163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:41.990430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] PATH_INFO: `/v3/projects/77868d2f7582430694fddb0422e13efd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:41.992281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] RBAC: Authorizing `identity:get_project(project_id=77868d2f7582430694fddb0422e13efd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:41.994099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:41.996948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 222/432] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/projects/77868d2f7582430694fddb0422e13efd => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:42.001650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.068078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495ea6a-0b09-4bcb-8ed2-e64d7394b2b1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.101572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.102244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.102667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.102942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.103799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] RBAC: Adding query filter params (name=member) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:42.104104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.105514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a3cc-edc3-49c0-b39a-ce8b45774491 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.113680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 211/433] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:42.114724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21d6b-71db-4ed2-95ce-ca4f24c00b8e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.115003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21d6b-71db-4ed2-95ce-ca4f24c00b8e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.115273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21d6b-71db-4ed2-95ce-ca4f24c00b8e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.116708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 212/434] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:42.119735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b21d6b-71db-4ed2-95ce-ca4f24c00b8e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.130314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358804d-0b45-4604-9c9e-241a793097a8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.174010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.175241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.175525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.175802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.177042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.179188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc2873-3765-4358-983f-cc1eeb1a0ef0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.186180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 223/435] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:42.186180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.186180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.186180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.202530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:42.203724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:42.225502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b21d6b-71db-4ed2-95ce-ca4f24c00b8e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.260101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.260803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.261070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.261342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.280273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 224/436] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:42.280273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.344836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.346297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.369075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d9f1-fa61-4a0f-8116-4e44c5d946a4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.402258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.402914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.403198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.403473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.404413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=77868d2f7582430694fddb0422e13efd) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:42.404756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.406193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.420447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 225/437] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=77868d2f7582430694fddb0422e13efd&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:42.420447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.455694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 213/438] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:42.464305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.542824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c21dd6-4a7d-4ed2-aafc-98713db1baae None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.579242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.580219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.580605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.580949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.591591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1846-0633-43fe-8cb4-98db1de149cc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.593962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:42.602588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:42.604975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:42.606079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 226/439] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:42.613618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.637875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.638781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.639142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.639397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.640686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.642160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.646112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 214/440] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:42.650469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.741491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb53-ec0c-4ed2-af2a-7e0d84ea887a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.773926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff87adb-f6fc-403a-b5cf-cfc723c7e22b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.790263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.791192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.791472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.791751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.792698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:42.793006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.794412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051e4db-d3fd-4029-9766-a826f1a65fea None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.802739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 227/441] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:42.804054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91543-330f-4d12-bbf9-786cb023494e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.804374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91543-330f-4d12-bbf9-786cb023494e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.804646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91543-330f-4d12-bbf9-786cb023494e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.810179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 228/442] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:42.811832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91543-330f-4d12-bbf9-786cb023494e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.819878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.820845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.821194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.821472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.822838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:42.825084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4330159-27ee-4b68-8e06-bbae917570fb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:42.826763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 215/443] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:42.828036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.829215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.829215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:42.845525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:42.846709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:42.918063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 216/444] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:42.918916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:42.927679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91543-330f-4d12-bbf9-786cb023494e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:42.967905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:42.968652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:42.968930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:42.969198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.025232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b440-6889-4832-8961-3762aaa930b5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.046775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.048278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.061737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.062411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.062691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.062964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.063798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=d2a59677d1e9405f8eeb3527dd367536, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:43.064108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.065654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.071796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 217/445] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=d2a59677d1e9405f8eeb3527dd367536 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:43.072445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.176796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 229/446] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:43.177852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.186410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a204d-6ca0-4e8a-9679-cf2b1fbf7cd3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.220971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.221809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.222089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.222355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.233624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:43.241809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:43.242792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:43.244077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:43.248212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 218/447] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:43.249205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.285204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8404b-a52c-4f82-a368-390700ca05e2 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.326570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.327698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.327977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.328283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] PATH_INFO: `/v3/projects/137a089466c04c34b8c9ec48a39d84b0` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.329478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] RBAC: Authorizing `identity:get_project(project_id=137a089466c04c34b8c9ec48a39d84b0)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.330921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.332941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 230/448] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/projects/137a089466c04c34b8c9ec48a39d84b0 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:43.338966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.401714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bb310-8070-464d-bfdc-7d58c4fb82b9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.445910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.446830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.447118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.447479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.448351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:43.448605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.450043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.459559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 219/449] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:43.460173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf9f3-a824-446e-b065-aeec49b518c7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.464789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.507721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.508971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.509828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.510110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] PATH_INFO: `/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.511944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] RBAC: Authorizing `identity:get_group(group_id=045c970ae3b74298a4f0cb4bd129bf0b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.512984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.514636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 231/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/groups/045c970ae3b74298a4f0cb4bd129bf0b => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:43.520447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.581113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ec5f8-dd09-4713-badc-1a4876cd547d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.614886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.615538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.615817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.616089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.617564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a8669-7625-4001-90ce-836e42fa9b23 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.654655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.655319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.655600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.655872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.656745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] RBAC: Adding query filter params (group.id=045c970ae3b74298a4f0cb4bd129bf0b, role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=137a089466c04c34b8c9ec48a39d84b0) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:43.657045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.658471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.664040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 232/451] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/role_assignments?group.id=045c970ae3b74298a4f0cb4bd129bf0b&scope.project.id=137a089466c04c34b8c9ec48a39d84b0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:43.683555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.685075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.790073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 220/452] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 00:39:43.796191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:43.903255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a456-6279-4ce0-aba3-cfa2c7e9248f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:43.935255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:43.936126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:43.936436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:43.936711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] PATH_INFO: `/v3/projects/3184670dd5ef4e518db1f41ed2dc877c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:43.937931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] RBAC: Authorizing `identity:get_project(project_id=3184670dd5ef4e518db1f41ed2dc877c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:43.939367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:43.941091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 233/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/projects/3184670dd5ef4e518db1f41ed2dc877c => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:43.947128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:44.049114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce1401-01a8-432a-8fbc-0d4804e43cf3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:44.092203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:44.093088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:44.093364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:44.093631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] PATH_INFO: `/v3/groups/e6f9eb6a96dc4273b614371e26aff828` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:44.094797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] RBAC: Authorizing `identity:get_group(group_id=e6f9eb6a96dc4273b614371e26aff828)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:44.096283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543dcd4-b903-4333-a5b8-391560a43efd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:44.097890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 221/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 12 00:39:43 2022] GET /identity/v3/groups/e6f9eb6a96dc4273b614371e26aff828 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:44.104027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:44.203175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4113bf-c06d-4e42-ae2e-26a873097677 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:44.236277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:44.236914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:44.237191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:44.237459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:44.238333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] RBAC: Adding query filter params (group.id=e6f9eb6a96dc4273b614371e26aff828, role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=3184670dd5ef4e518db1f41ed2dc877c) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:44.238634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:44.240070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a83bc-1ab7-4dd4-93ea-694c6f81a8fa None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:44.246096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 234/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 12 00:39:44 2022] GET /identity/v3/role_assignments?group.id=e6f9eb6a96dc4273b614371e26aff828&scope.project.id=3184670dd5ef4e518db1f41ed2dc877c&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:47.063023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f133667-633c-407c-b4cf-e26cd7163ba5 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.063628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f133667-633c-407c-b4cf-e26cd7163ba5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.063628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f133667-633c-407c-b4cf-e26cd7163ba5 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.065213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 222/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:47.075113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1135-6ed0-42e9-977f-6ab4de6ae776 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.075598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1135-6ed0-42e9-977f-6ab4de6ae776 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.075677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1135-6ed0-42e9-977f-6ab4de6ae776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.101247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-423e1135-6ed0-42e9-977f-6ab4de6ae776 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:47.102633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-423e1135-6ed0-42e9-977f-6ab4de6ae776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:47.197149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 235/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:47.241834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f133667-633c-407c-b4cf-e26cd7163ba5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:47.341726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7698-8e3c-4351-9942-800af5e0fc18 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.341911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7698-8e3c-4351-9942-800af5e0fc18 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.342190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7698-8e3c-4351-9942-800af5e0fc18 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.345023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 236/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:47.349088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.349373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.349652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.379020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:47.380214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:47.403167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f133667-633c-407c-b4cf-e26cd7163ba5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:47.475253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:47.476069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.476371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.476642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.496469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:47.497960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:47.519219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 237/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:47.528768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:47.529665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:47.530530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:47.530530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:47.530530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:47.530530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:47.530530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 223/460] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:47.541777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:47.549299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:47.655306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261841e-5142-415a-9b89-e8dd99f8e695 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:47.693458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:47.694553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.695048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.695551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.696705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:47.697229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:47.698866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc50f2c-0a2a-4112-8633-1f5390d17725 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:47.710924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 238/461] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:47.712729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b43d2-75fe-4a1f-901c-1478903a571d None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.713239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b43d2-75fe-4a1f-901c-1478903a571d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.713721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b43d2-75fe-4a1f-901c-1478903a571d None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.714268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529a5b8-ae57-4b8b-b42d-f7e4353cd26b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:47.715281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 239/462] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:47.716182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b43d2-75fe-4a1f-901c-1478903a571d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:47.749808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:47.751324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.752021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.752469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.768384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:47.770128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-eda2f31f-a0cd-4c09-acc0-6211731904c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:47.788309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:47.789380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:47.790826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:47.790826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:47.790826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:47.790826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:47.792674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 224/463] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:47.794209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.794704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.795251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.812470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:47.813995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:47.829840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b43d2-75fe-4a1f-901c-1478903a571d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:47.864643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:47.865315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.865592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.865864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.866578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:47.868125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560a7b-10f6-4fe3-b960-7745c762ae39 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:47.882944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 225/464] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:47.883975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:47.894622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 240/465] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:47 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:47.895769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93062e9-080a-4f20-a223-b6065c3b4c02 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.896052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93062e9-080a-4f20-a223-b6065c3b4c02 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.896340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93062e9-080a-4f20-a223-b6065c3b4c02 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.897555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 241/466] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:47.903795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:47.904131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:47.904396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:47.921851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:47.922960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:47.996619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 242/467] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:48.004835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.024622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e577-e3dd-4572-830b-43309f3fb045 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.069394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.071091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.071986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.072448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.073367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:48.073838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.075758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.089007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 226/468] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:48.090225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.130421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fbdfd-bd89-4147-bc38-06cf4e9239e7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.168425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.169607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.169894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.170171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.185048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.186441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:48.205384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:48.206261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:48.207105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:48.207105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:48.207105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:48.207105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:48.207105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 243/469] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:48.207786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.210768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919fd8-dba6-4dac-be30-98e3de15f46b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.256894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.258364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.259234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.259809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.275480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.277786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:48.299796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:48.300789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:48.301796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:48.301796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:48.301796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:48.301796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:48.303964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 227/470] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:48.309183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.320572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4b1cc-e99b-4e2b-ad0c-0ff78e938c1f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.365554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.366859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.367772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.368355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.369698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.372331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.402559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 244/471] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:48 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:48.404030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.428723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe9259-dd48-4aa1-9464-c70571e4c1b2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.473246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.474227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.474657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.475192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.476121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:48.476644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.478370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.491600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 228/472] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:48.501387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.520731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e6eee-c759-4b02-ad03-e7ba8cdb8a30 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.560136 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.560866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.561148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.561425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.562164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:48.562472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.564646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa21e4-e05c-4b74-b69f-8b613039b091 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.578023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 245/473] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:48.579232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2582e-f5fc-428c-9074-f329622792f8 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.579511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2582e-f5fc-428c-9074-f329622792f8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.579787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2582e-f5fc-428c-9074-f329622792f8 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.585443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 246/474] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:48.586429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2582e-f5fc-428c-9074-f329622792f8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.653358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a977e-4459-490b-b20f-1698f6216a1f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.692487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.693169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.693452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.693730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.694447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.695894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac5ee-fabb-4d26-8d89-74c60d1dcb14 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.701678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2582e-f5fc-428c-9074-f329622792f8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:48.727374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 229/475] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:48 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:48.729381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.729673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.729951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.753897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:48.754589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:48.754917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:48.755192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:48.755353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:48.755917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:48.756608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:48.757577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:48.785815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 247/476] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:48 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:48.847438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 230/477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:48.871643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:48.988861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e6c2-a024-459d-8b29-96d1f78d8898 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:49.028082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:49.028851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:49.029134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:49.029411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:49.030259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:49.031719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:49.039937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.sql.core [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 00:39:49.039937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 12 00:39:49.039937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [parameters: {'id': 'd6ef4a00b09144eb8c5aafbc2671f206', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 12 00:39:49.039937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=96826) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application conn.query(q) May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result.read() May 12 00:39:49.058544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return ref.to_dict() May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:39:49.060604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application next(self.gen) May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application yield resource May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application next(self.gen) May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.session.rollback() May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.force_reraise() May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise self.value May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application session.commit() May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 00:39:49.062494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._prepare_impl() May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.session.flush() May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._flush(objects) May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application compat.raise_( May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application flush_context.execute() May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rec.execute(self) May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 00:39:49.063537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application c = connection._execute_20( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:39:49.065098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application conn.query(q) May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result.read() May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:39:49.066737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [parameters: {'id': 'd6ef4a00b09144eb8c5aafbc2671f206', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:49.073168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 12 00:39:49.073982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:49.074810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 248/478] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 12 00:39:48 2022] POST /identity/v3/roles => generated 144 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:49.074874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:49.174902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91c395-5ba9-411d-b997-efee0d9c637b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:49.209698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:49.210762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:49.211176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:49.211674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:49.222602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-1109576a-785c-47b4-8154-ebfa28d1b783 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:49.231064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:49.232161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 00:39:49.233020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:49.234057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 231/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:39:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:49.239953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:49.379632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5fade-9bc4-4e6b-b077-3122d2a0a701 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:49.426783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:49.427770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:49.428224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:49.428724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:49.429651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:49.430133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:49.431854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd8f21-d60f-43aa-914e-e0a61e58d393 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:49.442822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 249/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:50.546036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190ed0-ba67-44fa-9b76-512846fc8c7e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:50.547026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190ed0-ba67-44fa-9b76-512846fc8c7e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:50.547507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190ed0-ba67-44fa-9b76-512846fc8c7e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:50.549428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 232/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:50.554564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:50.555446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:50.555921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:50.581238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:50.582760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:50.666036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 250/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:50.701562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190ed0-ba67-44fa-9b76-512846fc8c7e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:50.830490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190ed0-ba67-44fa-9b76-512846fc8c7e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:50.875458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:50.876280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:50.876561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:50.876834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:50.897081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:50.904819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:50.905716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:50.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 233/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:50.911843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.096103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4bdea-9b32-46ec-b5f1-3cfab04d5c77 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.147407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.148486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.148983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.149585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.150605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:51.151652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.153301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401ad5a-aa6c-480f-9b64-958a45bb2756 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.174998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 251/484] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:51.180294 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.199845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03159658-8a45-41cd-b70a-bc706e85de41 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.200159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03159658-8a45-41cd-b70a-bc706e85de41 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.200448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03159658-8a45-41cd-b70a-bc706e85de41 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.202193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 252/485] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:51.206933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.207214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.207489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.234691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:51.235864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:51.364472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 253/486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:51.366318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de44a78-b0d7-4e31-9d87-2739fefcdd49 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.400319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.408311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.409124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.409400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.409669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.435007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.436545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:51.466487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:51.467399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:51.468826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:51.468826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:51.468826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:51.468826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:51.468826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 234/487] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:51.473256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.537502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9459db-71b7-470b-9779-1a4a40e0f038 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.575910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.576769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.577052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.577325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.588266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:51.598303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:51.600441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:51.602441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 254/488] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:51.613231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f8eb-4625-4fb3-86ed-7cf1b2d70b49 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.684279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.684279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.684279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.684279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.685041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:51.685041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.685831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.707366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 235/489] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:51.708639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.734472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38aca-9739-449c-bd28-abeba235affe None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.768933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.769610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.769892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.770167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.770949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:51.771231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.772647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546e2c-5dfd-493b-a16c-bed86529ba55 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.780989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 255/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:51.785720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b91441-7cd2-4bb0-aef0-5495047a1eae None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.786011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b91441-7cd2-4bb0-aef0-5495047a1eae None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.786315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b91441-7cd2-4bb0-aef0-5495047a1eae None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.787592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 256/491] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:51.788939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a91da-1a50-4d2c-989a-3a962e549de9 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.790019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a91da-1a50-4d2c-989a-3a962e549de9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.790312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a91da-1a50-4d2c-989a-3a962e549de9 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.791532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 257/492] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:51.792403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a91da-1a50-4d2c-989a-3a962e549de9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.818850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d8ae3-2dda-4234-85c5-ec41384b72d9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.854266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.855084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.855342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.855612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.862514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.863957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-96a15b94-e95c-4cc1-8517-8b723201e9bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:51.880563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:51.881511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:39:51.882386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:51.884937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 236/493] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:51.886670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.887135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.887564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.904180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:51.904526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a91da-1a50-4d2c-989a-3a962e549de9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:51.905466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:51.938927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:51.939730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.940004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.940299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.950537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:51.951996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-7528c505-8e5e-4ce1-aca0-9b861d5e3826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:51.968571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:51.969470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:51.970320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:51.970320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:51.970320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:51.970320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:51.970320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 258/494] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:51.971277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:51.971557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:51.971843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:51.984132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 237/495] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:51.984132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:51.988065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:51.989313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:52.055172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 259/496] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:52.056055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.102265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7825-bce4-44e3-afc7-f49294b605a0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.138377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.139056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.139345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.139621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:52.140802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.142287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.159583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc951da1-ac79-4a67-8691-a4e78c4cdf32 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.178840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 238/497] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:39:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:52.179697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.197560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.198936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.199458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.199869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.200904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:52.201595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.203131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.219451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 260/498] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:52.219451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.288519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be5e91-3ffd-4116-838d-ac9072a292db None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.312311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5941de-38de-4817-8cb0-ae3cfca145d7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.322176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.323191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.323485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.323742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.333645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:52.341325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:52.342388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 239/499] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:52.343388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.352100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.352975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.353260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.353526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.364756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:52.372599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:52.373603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 261/500] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:52.374533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.460804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a463cd-12f7-4a9d-89d4-8ef911f8af59 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.477031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cfb98-50e1-4a5b-ae4a-b0fb17243f48 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.496594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.497253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.497540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.497815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.509333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.510132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.510414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.510699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] PATH_INFO: `/v3/users/glance` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.518226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.519654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:52.534675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:52.535612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 00:39:52.536919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:52.540575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 262/501] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:52.541205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.564739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.566229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.636112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f51759-693a-4c20-9d6d-0f79fd56fa7c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.659996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 240/502] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:52.660888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.677433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.679946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.680285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.680558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.681503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:52.681809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.683198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.693455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 263/503] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:52.694280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.761592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542299e2-e4ec-40cc-a246-3fccd4c6f2c8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.795210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.795872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.796150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.796426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c87-a924-4bc3-ab43-755147750561 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.796525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.797179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:52.797476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.798872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3073f3-beb7-408d-8f05-1085f803fa3d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.806958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 241/504] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:52.808233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595233-7db5-410b-9a21-2ab2c21ef632 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.808547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595233-7db5-410b-9a21-2ab2c21ef632 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.808826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595233-7db5-410b-9a21-2ab2c21ef632 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.810025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 242/505] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:52.810708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595233-7db5-410b-9a21-2ab2c21ef632 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.834492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.835152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.835424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.835696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.836657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:52.836972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.838400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.875542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 264/506] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:52.876201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:52.915975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595233-7db5-410b-9a21-2ab2c21ef632 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.949995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:52.950774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:52.951052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:52.951344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:52.961517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:52.963633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:52.971593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4e7bc-b0ca-413d-9c13-ed94f782e66a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:52.982858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:52.983696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:52.984600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:52.984600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:52.984600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:52.984600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:52.984600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 243/507] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:52.985326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.010223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.011025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.011301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.011570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.021649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.023050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0aefc3f3-4371-4d9c-8bfb-624b0ed0a5a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:53.040961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.042017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.042840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:53.042840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:53.042840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.042840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:53.042840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 265/508] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.043430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.043713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.043978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.060092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:53.061301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:53.089981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a37dd-20c4-4b2b-bb7f-abf0429a1560 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.121826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.122625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.122902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.123174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.127845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 266/509] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:53.129072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.134471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.135962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:53.159931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.160893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.161720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:53.161720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:53.161720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.161720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:53.161720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 244/510] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.161979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.229149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c98d7-c5e8-4975-9bb2-1537a65b6e86 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.262805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251181b8-f80d-4f5a-acbb-df74cb5f72f7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.263242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.263896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.264179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.264469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.294428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.295001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.295279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.295548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.296295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:53.296586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.297994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.311281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 245/511] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:53.312407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.334878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.337554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.410445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce025e-1578-404e-8ecd-d58d9a0cbdd2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.442157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 267/512] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:53.443116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.443236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.443737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.444015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.444292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.445019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:53.445331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.446802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.458209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 246/513] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:53.459013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.550158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460526b-3aa6-45b0-b428-494fcdded80f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.569023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92aa04-019c-492d-ae5c-aa7f6269f839 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.584444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.585232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.585505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.585768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.595675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.597151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.602188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.603292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.603558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.603826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.604574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:53.604885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.606255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:53.616589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.617637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.618491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:53.618491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:53.618491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.618491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:53.618491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 268/514] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.618926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.621692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 247/515] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:53.621692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.721913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1a3ce-b9fa-4b3b-b313-10a6ac5ad590 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.725760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee325-542f-4b46-8986-abff2e552825 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.757680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.758519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.758795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.759063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.761986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.762790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.763095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.763362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.767240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.768754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.775577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.777063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:39:53.788295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.789222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:53.790228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 269/516] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.790740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:53.796019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.796910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.797733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:53.797733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:53.797733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:53.797733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:53.797733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 248/517] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.798489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.906002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee4c0-f849-4914-9d6b-e36b00f4f912 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.910377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c6b7-71bf-4ad5-bdd7-4bc5816f3fad None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:53.946515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.947369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.947777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.948059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.953489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:53.953633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:53.953870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:53.954156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:53.960220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.960220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.962476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:53.963936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:53.972906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.974074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:53.974944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 270/518] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.975484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:53.980415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:53.981300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:53.982167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 249/519] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:53.982712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.080324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe863-6bfc-4b72-b83d-225ec38d4748 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.101433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c0ec-9ec9-490f-bc52-048162f67a88 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.123471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.124135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.124445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.124717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.125440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.125745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.127123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.135530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.136163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.136486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.136738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.137545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.137844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.139265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.140744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 271/520] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.141448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.162790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 250/521] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.163613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.252010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ddd68-3c36-46b9-aecb-fbdf8bea482a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.280126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4465-e88f-4fe5-960f-9a131e3bda46 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.288094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.288801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.289078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.289355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.290370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.290688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.292090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.303619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 272/522] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.304489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.326204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.326890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.327243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.327525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.328427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.328741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.330201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.356742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 251/523] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.357565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.433351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae0f3-0e72-4cb3-9e7c-48d51fdc2bdc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.465390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b053-001c-4842-bde4-8f5635205ed6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.472685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.473680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.473955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.474251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.475110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.475414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.476877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.495044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 273/524] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.499741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.505259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.505965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.506253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.506534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.507288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.509026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.530640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 252/525] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:39:54 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:39:54.531682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.611481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8745bde-98df-40aa-9169-95880cf822e4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.650462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544bcaa-892f-4efc-9209-17896112ecc6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.662394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.663250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.663634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.664011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.665402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:54.665922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.668320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.688626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 274/526] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:54.700287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.705966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.706803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.707086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.707363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.717648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.720449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:39:54.739396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:54.741088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:54.741920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 253/527] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:54.742371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:54.835700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a239-4ddd-4bf0-ba42-62af2a01a4f6 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.864926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188680b-c34f-4482-b53b-eada30f31ddc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:54.873544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.874333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.874695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.875043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.903995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:54.904785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:54.905064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:54.905336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:54.951928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.953334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:54.992795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:54.994323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.063355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 275/528] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:55.064239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.113988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 254/529] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:55.117043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.182034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53f7d-7834-4dbd-abd0-2b7f967e204b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.225630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26647995-0d46-483f-9d27-ea29888b18cd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.228797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.231133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.231133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.231133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.231133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:55.231133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.233317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.257777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 276/530] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:55.259521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.263703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.264117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.264423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.264753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.275436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.276966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:55.294948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:55.295847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:55.296717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:55.296717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:55.296717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:55.296717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:55.296717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 255/531] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:55.296988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.380122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3dbd7-7495-4a2e-b8b4-3aa9b0789b35 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.404681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88666-9ded-4817-ae72-19e567aa6bf4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.416238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.417611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.418090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.418561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.429298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.431378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.438752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.439311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.439591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.439860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.440721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=89fc09941e0b4aa29bbfe6d41e154f7f) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:55.441028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.442455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.448623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 256/532] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/role_assignments?user.id=89fc09941e0b4aa29bbfe6d41e154f7f&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:55.449510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:55.451732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:55.452819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:55.453669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:55.453669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:55.453669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:55.453669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:55.455894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 277/533] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:55.465373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.563327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0fa26-4f7d-45c1-94e9-fb520230a61e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.598468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.599201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.599482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.599762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.600560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:55.600861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.602469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.604529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267cd73-32d6-44db-806b-d48faaac0d8b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.614582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 257/534] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:55.620364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.640521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.641600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.642083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.642554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.643511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:55.644037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.645716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.661329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 278/535] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:55.669380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.729946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d05c42-6252-4721-84cf-fcdcdfc205c7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.771886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd6d1-f43c-469f-80da-936338f44a67 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:55.774879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.775709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.775993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.776290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.785109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.786593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:39:55.802218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:55.803219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:55.804043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 258/536] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:55.813438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:55.814844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:55.815659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:55.815933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:55.816420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:55.824754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:55.826706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:39:55.842484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:55.843778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:55.844652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 279/537] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:55.850526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:56.007509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab64d58-b95c-4c61-9f79-6590cc748eda None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:56.012585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3d20-c921-47d2-9108-4d8c53451269 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:56.069493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:56.070112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:56.070394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:56.070671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:56.071529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:56.072060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:56.073499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:56.074235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:56.074517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:56.074788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:56.075595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:56.075905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:56.077421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:56.080147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:56.098803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 280/538] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:56.100786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 259/539] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:56.104708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:56.117277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:56.208771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793008e2-d423-42e2-a0ac-7d12b5212edd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:56.242082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:56.242843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:56.243309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:56.243590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:56.248270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=d00e31520c5b421c8575fc063b1e6f74) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:56.248270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:56.248270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a573b-c065-463c-9f4c-799c309ff162 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:56.252147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 281/540] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:56 2022] GET /identity/v3/role_assignments?user.id=d00e31520c5b421c8575fc063b1e6f74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:56.307776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdec13-c915-4db1-ab48-bdf1f785eeeb None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:56.344227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:56.344850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:56.345133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:56.345403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:56.346252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=692dbeda127948b3812e6153241e7ce1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:56.346561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:56.348046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4664bc0-718b-409e-b79c-73109eb0c438 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:56.354857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 260/541] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:39:56 2022] GET /identity/v3/role_assignments?user.id=692dbeda127948b3812e6153241e7ce1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:57.126353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e00f9-8498-444a-a1dc-c52d5de27406 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.126907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e00f9-8498-444a-a1dc-c52d5de27406 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.126907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e00f9-8498-444a-a1dc-c52d5de27406 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.129035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 282/542] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:57.134771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f969-6e4f-4ec1-981c-1d097f50df7f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.135210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f969-6e4f-4ec1-981c-1d097f50df7f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.135376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f969-6e4f-4ec1-981c-1d097f50df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.161578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d679f969-6e4f-4ec1-981c-1d097f50df7f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:57.162776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d679f969-6e4f-4ec1-981c-1d097f50df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:57.250033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 261/543] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:57.297954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e00f9-8498-444a-a1dc-c52d5de27406 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:57.394962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe2d4-c54c-46f4-b862-583cd653d9bb None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.395589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe2d4-c54c-46f4-b862-583cd653d9bb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.396065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe2d4-c54c-46f4-b862-583cd653d9bb None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.397890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 262/544] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:57.403371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.403913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.404434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.416953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e00f9-8498-444a-a1dc-c52d5de27406 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:57.421901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:57.423311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:57.465827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:57.467037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.467519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.467988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.482671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:57.494987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:57.495890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:57.496735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:57.503537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 283/545] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:57.509970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:57.540581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 263/546] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:39:57.601208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:57.660742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4e3a4-d168-419b-8b74-676276d897af None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:57.734619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:57.735295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.735587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.736998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.737764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:57.738068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:57.739438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:57.758165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 284/547] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:57.763978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:57.767930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fc724-db3f-46d1-8d9d-80607f7daea7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:57.854451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:57.855655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.856139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.856632 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.869322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba064a-21c7-4cdb-a48f-4d5d6d45ac7f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:57.889343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:57.901981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:57.902946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:57.903748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 264/548] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:57.911048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:57.915781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:57.917504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:57.917788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:57.918127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:57.942571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:57.944586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:57.966170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:57.967101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:57.967926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:57.967926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:57.967926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:57.967926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:57.967926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 285/549] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:57.975059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.095261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77226f2d-d0ad-45b6-b072-abbf3059e0a2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.109166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b53d99-4873-4b0f-bcab-0ad02616a824 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.131277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.131962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.132402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.132687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.133722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:58.134103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.135652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.144422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 265/550] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:58.150430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.177331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.177878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.177980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.178178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.178938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:58.179210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.180947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.201618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 286/551] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:58.213643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.259287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cacef-2eda-463f-a39b-10e0a35afa5c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.293549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.294634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.295116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.295536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.307515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.309258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.319617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b6289-5a06-4e01-84fa-2302c5d93dc4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-5423d237-9121-4830-b053-524ff7d8dd29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:58.327057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:58.328104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:58.329182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:58.329182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:58.329182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:58.329182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:58.331029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 266/552] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:39:58.336597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c714-8a7c-41d1-bdb6-32fb03e81b31 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.337061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c714-8a7c-41d1-bdb6-32fb03e81b31 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.337486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c714-8a7c-41d1-bdb6-32fb03e81b31 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.339271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 267/553] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:58.340784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c714-8a7c-41d1-bdb6-32fb03e81b31 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.359106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.359907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.360184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.360479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.367803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.369539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-74e48d00-aa57-44f3-a3c3-fa068093c0e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:58.384968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:58.386122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:39:58.387224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:58.391173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 287/554] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:58.392220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.392524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.393017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.409604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:58.411143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:58.453624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c714-8a7c-41d1-bdb6-32fb03e81b31 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.484506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 288/555] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:58.485435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.489310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.490153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.490611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.491052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.491921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:58.492442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.494138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.507609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 268/556] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:39:58.513501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.597694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb58589-4fe1-44f8-8780-6548dee2766e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.616486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9948b5f-0c1d-4a68-92e3-226efdd4878b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.631306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.631989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.632280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.632643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.633460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:58.633768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.635319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.657141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 289/557] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:58.657141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.660152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.661159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.661610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.662025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.671008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.672902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:39:58.689329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:58.690567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:39:58.691616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:58.693991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 269/558] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:58.699518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.759958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2daf9-63f4-4257-876e-bac5cf14ed0e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.793546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.794360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.794637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.794908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.805085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:58.811415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb633-9277-4ddc-b006-2f35935e4d39 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:58.813146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:58.814178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:58.815064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:58.819171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 290/559] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:58.820054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:58.850855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.851784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.852286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.852706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.921065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8e054-b250-4f2f-af87-f0d51f1e845e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:58.931006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.932501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082e8d0-5aa6-468d-ab74-f9e53b9fb2b9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.955492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:58.956301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:58.956590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:58.956891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:58.957722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:58.958037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:58.959481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:58.977321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 291/560] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:58.978175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.045081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 270/561] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:39:59.047383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0187b-440f-4c0d-abe5-db933ed93aef None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.047820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0187b-440f-4c0d-abe5-db933ed93aef None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.048219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0187b-440f-4c0d-abe5-db933ed93aef None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.050142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 271/562] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:59.056838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0187b-440f-4c0d-abe5-db933ed93aef None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.093588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b57ce0-d077-440c-8e16-f2ba8b5f58a8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.127792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.128848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.129121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.129388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.130131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:59.130422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.131798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.139414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 292/563] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:59.140189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.173240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0187b-440f-4c0d-abe5-db933ed93aef None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.206728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.207520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.207795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.208064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.218026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.219486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.231428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b193fb2-a985-4fa8-9ddf-5d1d7b0631c3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-38754874-b220-4bfa-8bc6-d43c00f56101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:59.236816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.238038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.238959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:59.238959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:59.238959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.238959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:59.238959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 272/564] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.239258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.239415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.239685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.256424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:59.257601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:59.266476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.267270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.267547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.267816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.278041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.279485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5307e2f9-b257-4aeb-b0a1-487b740c45d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:59.296577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.297497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.298516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:59.298516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:59.298516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.298516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:59.298516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 293/565] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.298897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba9558-142c-4ba5-8312-17e8b168ab71 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.299223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba9558-142c-4ba5-8312-17e8b168ab71 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.299449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba9558-142c-4ba5-8312-17e8b168ab71 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.300699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 294/566] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:39:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:39:59.301380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba9558-142c-4ba5-8312-17e8b168ab71 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.326977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 273/567] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:59.327819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.403254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba9558-142c-4ba5-8312-17e8b168ab71 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.435392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.436181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.436338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56def9-cda7-4566-8e35-6b387ba0fe31 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.436493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.436753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.447514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.448988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:39:59.465885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.466858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.467720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:39:59.467720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:39:59.467720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:39:59.467720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:59.467720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 295/568] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.468062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.469367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.470033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.470317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.470591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.471321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:59.471847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.473272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6297-6042-4fe6-b3d8-68c107321d07 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.484430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 274/569] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:59.485712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.486000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.486274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.502326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:39:59.504064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:39:59.567354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 275/570] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:39:59.567602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41fb43-1a61-42e0-9740-daf4a745339b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.567975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.600904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.601575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.602133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.602133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.603018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:59.603323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.604733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.616072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 296/571] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:59.616892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.673778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10761c-e014-47c4-a10f-b49c14c13336 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.707645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.708456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.708732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.709002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.713044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae1742-e05e-4d9f-b455-0759ee5100bf None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.718951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:59.726565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.727717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 276/572] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.729043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.747945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.748634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.748914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.749174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.749889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:39:59.750192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.751562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.762741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 297/573] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:39:59.763517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.827248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a01dc-fded-49b0-b244-6b897de6f9c2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.860143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.860557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7c5ee-b832-4435-9576-884095f6d01f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:39:59.860974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.861251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.861517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.868162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:39:59.869847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:39:59.884560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.885554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 277/574] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.886521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.899045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:39:59.899860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:39:59.900142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:39:59.900455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:39:59.910453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:39:59.918177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:39:59.919197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:39:59.920232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 298/575] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:39:59.921498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:39:59.983638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93528b-906f-42f7-89aa-a6f0c940cab8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.016663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55eb07-2539-4698-9c68-68eea6e262d8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.017527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.018327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.018616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.018888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.025663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.027150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:00.040937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:00.042091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 278/576] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:00.042975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.050195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.050868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.051152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.051425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.052180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.052514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.053880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.061738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 299/577] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:39:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.062515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.138134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3feb7-43f5-4dad-b208-857f8544f2e9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.159410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8739c-fec4-4163-b882-4e216abd1a41 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.169766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.170565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.170845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.171114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] PATH_INFO: `/v3/users/glance` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.178396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.179828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.192656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.193320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.193606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:00.193769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:00.194842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 00:40:00.195927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:00.196737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.196737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.196737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.196929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 279/578] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:00.196929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.197072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.208478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 300/579] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.209287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.294753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a326dd-2aa3-4821-ba06-6ea57a2d4242 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.303236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1259b3-7063-48be-bf80-b66d08740895 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.327816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.328509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.328798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.329086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.329813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.330117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.331487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.335887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.336801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.337079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.337350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.338132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.338436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.339473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 280/580] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.339804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.340444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.355930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 301/581] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.355930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.441180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138611f-8a05-429a-90a8-8438bfbb20d6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.445475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ebbec-6062-42bf-9a78-991af7228573 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.473065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.473864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.474145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.474417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.476148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.476837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.477148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.477419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.478247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.478549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.479992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.484893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.486315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.497076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 302/582] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.497848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:00.502758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:00.504007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:00.504928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:00.504928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:00.504928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.504928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:00.504928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 281/583] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:00.505208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.590150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f8c6-e9e0-4b20-99d8-a1b681aa6684 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.603971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1c37f-3986-4d06-95c1-113d3c93e36b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.623291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.624205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.624512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.624783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.636918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.637730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.638009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.638278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.639036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=736e16d1bc05411faea9c943f930acb1, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.640512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.648627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.650208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.653483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 303/584] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:00 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:00.654289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:00.670594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:00.671684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 282/585] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:00.672993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.761954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09404d7e-722d-42f7-99ff-4a8c10cfc680 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.782936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ba497-dca8-4d6d-afd0-f01495f8450c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:00.798153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.798836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.799110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.799377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.800228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=86ce304b33114f83a8fa63a0227aeef5) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:00.800591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.802076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.820290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 304/586] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/role_assignments?user.id=86ce304b33114f83a8fa63a0227aeef5&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:00.820290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.831039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:00.831956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:00.833343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:00.833343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:00.848337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:00.851412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:00.873271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:00.874230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:00.875087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:00.875087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:00.875087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:00.875087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:00.875087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 283/587] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:00.875873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:00.969040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f5ad-ddfa-48b7-9b15-3cddd9592a2e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.000638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9132-9097-46dd-96b7-4d9e87831dd2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.006999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.007670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.007945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.008325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.009853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.010162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.011620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.023559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 305/588] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.025140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.043993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.045110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.045604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.046074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.047025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.047521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.049142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.072625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 284/589] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.073927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.132298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c97f-7946-48c4-ade2-44dc6d866f49 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.165906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.166585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.166869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.167147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.167880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.168192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.169602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.188799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 306/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.189715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.194073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca882d-fdde-4b72-a58c-2b51e226cce9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.229739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.230931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.231425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.231994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.239989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.241765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:01.257193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:01.258242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:01.259156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:01.261563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 285/591] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:01.262685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.303200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6310b-0c68-4053-8ef1-ec353a629832 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.346285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.348171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.348859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.349384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.359477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.361722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.367999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6447a2-aecf-40d7-8ce7-4b4c57c6d2c0 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:01.381506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:01.382459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:40:01.383433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:01.390579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 307/592] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:01.390579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.410878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.411526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.411811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.412087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.418943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.420401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:01.435315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:01.436775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:01.437886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 286/593] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:01.441139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.514152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91170-dfe5-4c27-8059-cc105427645e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.545007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dd6ad-00fd-4e56-9629-e6e02aae064d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.556401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.557620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.558114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.558711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.560286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.560929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.563010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.579657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.580395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.580625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.580951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.581808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.582170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.583659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.585341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 308/594] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.586670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.611244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 287/595] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.612263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.713385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8459-79e4-4a64-9516-d6c4de638b51 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.717170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e21c45-aa44-4395-8e4b-bb10c1b27772 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.751264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.752002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.752318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.752601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.753902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:01.754216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.755820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.757170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.758785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.759250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.759708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.771452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.773196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.775019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 309/596] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:01.776651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:01.795260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:01.796233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:01.797173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:01.797173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:01.797173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:01.797173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:01.800661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 288/597] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:01.801896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.886327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431ec29-ffe8-4972-b963-4c4dfb58e6e4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.919539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.920286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.920598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.920874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.924552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3de3-63bf-451d-9a03-a29e07fe2d06 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:01.932303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.933960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:01.951518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:01.952513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:01.953398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:01.953398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:01.953398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:01.953398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:01.956715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 310/598] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:01.957564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:01.966573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:01.968204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:01.968738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:01.969216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/89fc09941e0b4aa29bbfe6d41e154f7f/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:01.986452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=89fc09941e0b4aa29bbfe6d41e154f7f, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:01.988053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.008361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 289/599] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:01 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/89fc09941e0b4aa29bbfe6d41e154f7f/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:02.010013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.154757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed86729-3056-4a4a-a14b-5294b5fbc675 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.177749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a54aa-fcfd-471f-982c-f3452332d7dc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.223301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.225352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.226874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.227729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.228303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.229295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:02.229806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.231431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.233191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.233428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.233715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.234482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:02.234796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.240286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.251401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 311/600] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:02.255595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 290/601] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:02.261236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.262079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.385551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4466b4c-0aa1-40d0-8646-d9dd172a4e0a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.399550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a856a3-f688-40e7-94b4-94024568c6f0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.421488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.422617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.423077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.423524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.432640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.434663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.441281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.448281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.448281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.448281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:02.459041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:02.459944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:02.460942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:02.461941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.463084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.471475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 291/602] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:02.472694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:02.499991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:02.500972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:02.501838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:02.504857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 312/603] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:02.511832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.612056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f5355-e35f-412b-bf15-c9796c860a01 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.646761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.647440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.647721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.647997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.648874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:02.649241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.650810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.663570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 313/604] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:02.669282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.680018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20074a11-ff37-4e88-a95c-44e63eef64d5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.774086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d3c3a-17a6-475f-807d-a00d0c8527cd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:02.778355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.779038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.779321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.779597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.780423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:02.780739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.782196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.794899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 292/605] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:02.808407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:02.834650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:02.835740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.836019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.836503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/692dbeda127948b3812e6153241e7ce1/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.863959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=692dbeda127948b3812e6153241e7ce1, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:02.865442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fd008-7179-4a73-9862-c75808657ecd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:02.885977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 314/606] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:02 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/692dbeda127948b3812e6153241e7ce1/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:02.982537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a061-8c5c-48cb-b092-988774e6218c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.982735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a061-8c5c-48cb-b092-988774e6218c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.982991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a061-8c5c-48cb-b092-988774e6218c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.984583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 315/607] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:02.989859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc26e0-cbf1-42c6-8ae9-3426b98472b3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:02.990141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc26e0-cbf1-42c6-8ae9-3426b98472b3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:02.990419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc26e0-cbf1-42c6-8ae9-3426b98472b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:02.997128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459bb05-e630-4562-a7de-e1626451e17f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.032448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7efc26e0-cbf1-42c6-8ae9-3426b98472b3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:03.033692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7efc26e0-cbf1-42c6-8ae9-3426b98472b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:03.042951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:03.043945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.044231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.044520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/d00e31520c5b421c8575fc063b1e6f74/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.062069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=d00e31520c5b421c8575fc063b1e6f74, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:03.063546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:03.087371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 293/608] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:02 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/d00e31520c5b421c8575fc063b1e6f74/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:03.157747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 316/609] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:40:03.183297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.308466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92a50-7043-455e-ac1a-8d64364bf966 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.317057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb094-ee38-465f-8c5a-cff53d22c9fb None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.317361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb094-ee38-465f-8c5a-cff53d22c9fb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.317614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb094-ee38-465f-8c5a-cff53d22c9fb None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.319239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 317/610] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:03.329943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.330550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.331027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.349851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:03.350674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.350955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.351229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.353790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:03.355648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:03.363214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:03.371758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:03.372747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:03.373923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 294/611] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:03.379577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.448272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 318/612] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:40:03.465924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.485500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb084-b7b5-4ee7-a9c1-2c7be8554416 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.525404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:03.526087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.526370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.526649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.527410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:03.527716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:03.529132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:03.540912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 295/613] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:03.547095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.684072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed4d9-f5dd-4606-a022-767f0d3a8387 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.735452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4f950-34df-498f-91ce-83c29e4d270c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.752544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:03.753730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.754014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.754288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.770341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:03.782883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:03.783815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:03.785124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 319/614] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:03.791186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:03.792152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:03.792462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:03.792736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:03.793207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.803397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:03.804960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:03.842488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:03.844631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:03.846733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:03.846733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:03.846733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:03.846733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:03.848764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 296/615] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:03.854279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:03.966119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5ad6b-4784-4fde-a1f9-af4c8686b3f1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:03.988110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43b6c7-5433-43ac-b131-fd30861add08 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.017028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.017719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.018002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.018276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.019034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:04.019339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.022500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.023170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.023454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.023762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.024528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:04.024837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.025006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.026208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.038610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 297/616] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:04.043717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 320/617] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:04.044110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.049966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.145288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692492c0-f49b-4410-a686-28b765adb550 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.193004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.193834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.194119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.194401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.209417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.210886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.217303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3cecc-e2ab-4474-ace8-111b9ca50525 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:04.236599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:04.237586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:04.238506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:04.244696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 298/618] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:04.250223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.255344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.256411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.256905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.257390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.269782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.271906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:04.291021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:04.291957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:04.292878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:04.292878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:04.292878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:04.292878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:04.299817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 321/619] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:04.301670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.451146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae7f41-d869-48dc-8c70-306a207f5920 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.467526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ed562-1591-4a93-aaf6-79d76990e023 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.513259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.513934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.514221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.514493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.515221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:04.515525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.516922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.532216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.532943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.533224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.533496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.534369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:04.534739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.536283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.542980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 322/620] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:04.548811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.572150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 299/621] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:04.577743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.719009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec46cd-cc8a-47ef-86b4-782324800774 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.753797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.754608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.754891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.755168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.762791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.764275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.766188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c96f9-bde8-4dfb-af54-1394b1803d6a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:04.779084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:04.781652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:04.783846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 323/622] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:04.792189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:04.839772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.840458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.840745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.841024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.921144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.922621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f53bb-bc6e-4b7b-8be6-877eb694f241 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.923648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3687c-45f4-4165-9024-0b04f5e4e3c3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:04.956590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:04.957175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.957435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.957702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.958553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:04.958842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:04.960288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c5fd8-1939-47da-92dc-b21b92fbfb98 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:04.978168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 324/623] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:04.979490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefac3-59fd-42c6-9262-19fa76f61368 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:04.979774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefac3-59fd-42c6-9262-19fa76f61368 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:04.980044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefac3-59fd-42c6-9262-19fa76f61368 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:04.985409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 325/624] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:04.986240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefac3-59fd-42c6-9262-19fa76f61368 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.026613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 300/625] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:05.028041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.028360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.028643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.045464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:05.046677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:05.085531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefac3-59fd-42c6-9262-19fa76f61368 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.116372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 301/626] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:05.117357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.129559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.130381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.130661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.130927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.154413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.154413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:05.171819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.173960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 326/627] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.176128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.234962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32092c-da7a-4f5c-8a1c-933ec2c5390f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.269649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.270455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.270732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.271000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.281769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.283332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.286036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532d60-ceb4-4c6d-84b0-561b9b8b6305 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:05.300985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.303076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 302/628] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.305407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.327609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.329082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.329470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.329975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.343250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:05.352390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.354515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:05.356553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 327/629] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.364428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.426626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afbad-fea7-4f1a-8566-44c999785519 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.461010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.461810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.462088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.462358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.463083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:05.463382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.464821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.473993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb504-49e4-4168-8e43-97b2d743928f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.476538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 303/630] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:05.477542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.512367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.513247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.513528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.513803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.514725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:05.515041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.517511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.535755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 328/631] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:05.536675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.580622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892898f-3ddd-4583-be0d-7686439847df None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.614909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.615578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.615862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.616137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.616909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:05.617211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.618826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.627174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 304/632] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:05.628487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.641466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dadbe-9a02-4629-84fd-8036aaa85833 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.674412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.675439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.675595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.675762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.682521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.683975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ed0582d3-b0bb-4f0c-b003-d8a8aca871ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:05.697575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.698487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:05.699475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 329/633] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.699969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95a1a4-d249-4947-82e7-666a7b3d9a4d None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.700314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95a1a4-d249-4947-82e7-666a7b3d9a4d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.700501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95a1a4-d249-4947-82e7-666a7b3d9a4d None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.701881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 330/634] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:05.703892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95a1a4-d249-4947-82e7-666a7b3d9a4d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.724727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2e7f-a4f6-40cc-bc9f-c8f29136be41 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.758898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.759729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.759998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.760281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.767009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.768447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-0471d6d7-ffe1-4490-b5f9-377898dc2393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:05.782392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.783281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:05.784140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 305/635] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.784860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be23b-dacf-43bb-891e-e1e4f99afd6e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.785068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be23b-dacf-43bb-891e-e1e4f99afd6e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.785335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be23b-dacf-43bb-891e-e1e4f99afd6e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.786577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 306/636] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:05.787261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2be23b-dacf-43bb-891e-e1e4f99afd6e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.809826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95a1a4-d249-4947-82e7-666a7b3d9a4d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.844031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.844859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.845138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.845437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.858946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.860557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-6d5ef116-e280-4ec6-a9ec-29f63f1c8cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:05.877485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:05.878849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:05.879719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:05.879719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:05.879719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:05.879719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:05.879719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 331/637] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:05.880333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15d9b2-5903-42b3-ac96-383e663e82b0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.880465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15d9b2-5903-42b3-ac96-383e663e82b0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.880735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15d9b2-5903-42b3-ac96-383e663e82b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.897023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2be23b-dacf-43bb-891e-e1e4f99afd6e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:05.897361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4f15d9b2-5903-42b3-ac96-383e663e82b0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:05.898535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4f15d9b2-5903-42b3-ac96-383e663e82b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:05.929369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:05.930183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.930528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.930876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.931658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:05.931962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:05.933430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:05.944279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 307/638] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:05.945073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:05.965486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 332/639] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:05.966733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:05.967014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:05.967281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:05.984532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:05.985706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:06.040906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114db9ab-fd5b-48a0-b214-8ee01d4c5b2e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.049482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 333/640] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:06.050316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.073108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.074053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.074477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.074903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.075685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.075991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.077482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.089800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 308/641] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.090562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.147312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30735b8-06b1-416b-8352-87f91f687ceb None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.183119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.183785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.183884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd7288-1326-4724-ae55-9be3efa88b96 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.184067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.184367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.185607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.186122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.188495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.201383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 334/642] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.202233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.218462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.219387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.219656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.219923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/86ce304b33114f83a8fa63a0227aeef5/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.234472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=86ce304b33114f83a8fa63a0227aeef5, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.235985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.250625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 309/643] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:06 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/86ce304b33114f83a8fa63a0227aeef5/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:06.264493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.310543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530504af-ee1c-44af-a06b-6f752f44a61f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.353913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.354752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.355032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.355305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.357603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211039-18ff-40fd-b706-9b0584e17930 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.366666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:06.375270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:06.376596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:06.377479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 335/644] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:06.379101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.399504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.400815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.401126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.401372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.412637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:06.421172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:06.422389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:06.423490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 310/645] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:06.424360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.491613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8426e-40ca-4b41-8140-17090d86d2e2 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.528612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.529288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.529567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.529842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.530677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.530987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.532463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.535661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9c63-f5f3-403e-ba00-8bda8a6b5db3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.538439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 336/646] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.539325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.582671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11aed-1684-4047-8010-bac32317536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.583837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11aed-1684-4047-8010-bac32317536b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.584329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11aed-1684-4047-8010-bac32317536b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.584771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11aed-1684-4047-8010-bac32317536b None admin] PATH_INFO: `/v3/users/glance` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.595736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba11aed-1684-4047-8010-bac32317536b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.598044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba11aed-1684-4047-8010-bac32317536b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-bba11aed-1684-4047-8010-bac32317536b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:06.618730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:06.619950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 00:40:06.621136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:06.623813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 311/647] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:06.625028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11aed-1684-4047-8010-bac32317536b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.668699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9414df-febd-4a95-8b4f-ec3f46b189e1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.707595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.708501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.708890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.709271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.710148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.710574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.712580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.722070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 337/648] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.723617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.754648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11aed-1684-4047-8010-bac32317536b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.791767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.792490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.792771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.793052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.793816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.794126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.795511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.808275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 312/649] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.808275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.847776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860b0f-7ac6-4f55-8593-2d661131fc43 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.886102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.886784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.887072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.887345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.888188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:06.888530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.889964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:06.912460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 338/650] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:06.913342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:06.928844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f15811-9e0e-4b59-9fc2-484e835ae076 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:06.971857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:06.972706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:06.973323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:06.973609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:06.987986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:06.989500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:07.011228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:07.012383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 313/651] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:07.013298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.028025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b431-d50a-4e73-911f-b9fd61183bf1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.062746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.063644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.064029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.064342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.079466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.081102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:07.099768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:07.100946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:07.102099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:07.102099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:07.102099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.102099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:07.103467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 339/652] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:07.104490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.147867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02251d45-d9f8-4171-8173-0ee593617bf3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.189556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.190783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.191217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.191638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.211334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6153ac-4ecc-4ae8-8627-6544c4d573b1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.245563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.246375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.246849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.247245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.248095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:07.248745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.250096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.266454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 340/653] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:07.266454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.269531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.271217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.360371 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716ef5d-6e07-45a2-b999-00b434c934ff None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.374148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 314/654] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:07.375450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.396158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.396861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.397138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.397407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.398127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:07.398428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.400694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.412718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 341/655] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:07.413542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.478154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e7b9-7de5-4eda-a697-955d090849f4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.512763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405be1-1858-4e06-8dda-29f6cfb832c5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.513943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.515003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.515483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.515955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.523900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.525888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:07.541344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:07.542755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 00:40:07.543692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:07.551012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 315/656] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:07.551675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.552433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.552543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.552865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.553137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.564399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.565993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:07.583936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:07.585306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:07.586625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:07.586625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:07.586625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:07.586625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:07.590033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 342/657] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:07.590982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.658825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd1e9e-62cf-4360-b682-bf075b01e98e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.687689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e209d-78c6-4415-9cd0-9c0497967993 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.693474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.694596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.695098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.695630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.703043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.704755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:07.719567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:07.720639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:07.721570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:07.722110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 316/658] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:07.722718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.725118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.725781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.726163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.726433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.728093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:07.728093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.729450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.752285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 343/659] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:07.752285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.849928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c26dfa-a242-4469-8ac9-7755d0042b27 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.865564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16686b1e-b4d5-474b-9dd9-ec42b71080ab None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:07.886117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.886569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.886851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.887119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.887868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:07.888143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.889538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.907171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 317/660] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:07.907171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:07.918577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:07.919801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:07.920238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:07.920956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:07.921975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:07.922592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:07.924900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:07.947735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 344/661] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:07.949314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.008730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf066-65ac-435a-a250-28ceb95a655c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.041760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.042647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.043176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.043545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.071940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceff624-60f8-4441-a594-5b26a6749f53 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.117172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.118697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.118987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.119507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.119792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.120070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.129622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.131437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:08.147838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:08.149055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:08.149973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 345/662] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:08.150485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.228079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 318/663] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:08.229403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.273312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5d471-630e-4e42-b1f8-928d68116574 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.317999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.318681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.318968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.319248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.343613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abc838-898f-4f5f-a7c5-8865fabf24b6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.388179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.388589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.389269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.389602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.389889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.390849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:08.390966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.391160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.392708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.412281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 319/664] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:08.412281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.511931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 346/665] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:08.511931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.524380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6f88-0269-42e0-b026-e58efe4d2724 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.558021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.560059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.560629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.561074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.572882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.574343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-bff0a872-1b0f-43b8-be14-d9f860ee5a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:08.591214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:08.592203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:08.593138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:08.593138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:08.593138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:08.593138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:08.593138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 320/666] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:08.593887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ed67-8bc2-4e92-ba54-e8c07951f8ce None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.594059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ed67-8bc2-4e92-ba54-e8c07951f8ce None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.594300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ed67-8bc2-4e92-ba54-e8c07951f8ce None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.599843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 321/667] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:08.599843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ed67-8bc2-4e92-ba54-e8c07951f8ce None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.611234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d66140-86df-4b2b-9073-2161c2518974 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.644526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.645213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.645494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.645771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.646604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=89fc09941e0b4aa29bbfe6d41e154f7f) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:08.646913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.648352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.654027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 347/668] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/role_assignments?user.id=89fc09941e0b4aa29bbfe6d41e154f7f&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:08.654800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.733051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ed67-8bc2-4e92-ba54-e8c07951f8ce None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.762880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c85c-688a-44fd-8e02-815c9f8f4eb5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:08.780743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.782070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.782904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.783387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.800301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.802174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:08.802480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.802896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.803184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.803486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.804204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:08.804549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:08.806196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16135774-2309-45a5-b973-7d3d17fe6685 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:08.819548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 348/669] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:08.821030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:08.821318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:08.821591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:08.826262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:08.827369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:08.828481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:08.828481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:08.828481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:08.828481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:08.836363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 322/670] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:08.836363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.847131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:08.848393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:08.938532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 349/671] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:08.939493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:08.973860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad8cb6-ca9f-4ab0-a247-f6b0f550799b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.012229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.013409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.013409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.013596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.021133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.022586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:09.037247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:09.038323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 323/672] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:09.039258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.050393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200c2f5-2ac4-43c5-b8c3-430b5c0a8fa0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.088463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.089134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.089415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.089710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.090436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.090738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.092093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248943c-1e68-40c5-9b33-2f96c7bc9a72 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.103643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 350/673] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.104946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c7fe-be1f-4221-b3e6-c08d5025e49b None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.105229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c7fe-be1f-4221-b3e6-c08d5025e49b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.105502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c7fe-be1f-4221-b3e6-c08d5025e49b None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.109842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 351/674] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:09.109842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3c7fe-be1f-4221-b3e6-c08d5025e49b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.155744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca962-efe0-4fd4-bcfc-7c7c1de61171 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.189410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.190307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.190588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.190853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.201508 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:09.209451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:09.210388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:09.216570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:09.217355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3c7fe-be1f-4221-b3e6-c08d5025e49b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.217452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 324/675] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:09.217452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.248575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.249518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.249796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.250067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.250861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.251168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.252646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710897-211b-4dc9-a16c-14e9ca65d7a4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.266329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 352/676] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.267530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.267813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.268083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.284570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:09.285736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:09.326807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16abf5b-3c56-4ffe-9963-7376907c7729 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.354399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 353/677] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:09.355251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.365438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.366620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.366898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.367216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.376175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.377798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:09.392775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:09.393681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:09.394577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:09.396567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 325/678] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:09.398072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.454882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf1e37-1a9e-446c-a22e-48d367565a2c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.488937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.489673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.489956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.490280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.491032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.491341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.492919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.501214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35d060-d4b1-4f2f-82f3-fa5dedac3208 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.506571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 354/679] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.507320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.535085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.535669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.535950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.536229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.537106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=692dbeda127948b3812e6153241e7ce1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.537419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.538856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.556636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 326/680] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/role_assignments?user.id=692dbeda127948b3812e6153241e7ce1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.556636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.614633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8258b47-b579-4be3-b764-01634aea706b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.650593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.651422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.651705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.651978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.663135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:09.671227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:09.672278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:09.673243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 355/681] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:09.673934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.693414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e73825-2675-4fbd-969d-d4752a69e6ce None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.737285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.738239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.738527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.738807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.739773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.740088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.742189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.758576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 327/682] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.759831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.790096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c797-9487-4103-8411-9bf714cca711 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.829304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.830480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.830766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.831035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.844851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.846411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:09.864510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:09.865634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:09.866525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:09.866525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:09.866525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:09.866525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:09.866525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 356/683] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:09.866822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.888081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c80b3-622a-4cd4-9593-ef7f858de7d1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:09.923276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:09.924321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:09.924756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:09.925196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:09.926144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:09.926620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:09.928224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:09.938265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 328/684] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:09.943751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:09.994755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c45454-8907-4a12-b88f-03236d7ec335 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.028468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.029282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.029663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.030038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.030998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=d00e31520c5b421c8575fc063b1e6f74) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:10.031427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.033071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.042546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 357/685] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/role_assignments?user.id=d00e31520c5b421c8575fc063b1e6f74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:10.043671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.052778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d789-f1df-436e-930e-cae0436fb64b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.096379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.097421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.098178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.098615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.099760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:10.100314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.103063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.123382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 329/686] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:10.129695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.264017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c286a-7225-4727-89c7-3adb77b0a099 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.308203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80fddd-773e-41cb-9691-354974e8dff0 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.320241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.321333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.321464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.321683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.337872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.339341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.350497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.351596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.352112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.352682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:10.359215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:10.360132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:10.361087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:10.361087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:10.361087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:10.361087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:10.361087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 358/687] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:10.363333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.365108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.366274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:10.384468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:10.385490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:10.386527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:10.389885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 330/688] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:10.395693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.499531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abae9a-fedc-408b-a5be-200a25178e98 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.500337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72443d4-8066-4c89-a9cb-bfb396b6cb47 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.538163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.539224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.539708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.540226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.540665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.541407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.541729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:10.541835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.542092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.542496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.542955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:10.543442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.545090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.545628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.568408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 359/689] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:10.571065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 331/690] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:10.576903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.577047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.772499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1606-d147-40ca-b67c-78afa2fb90ff None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.775063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2713-e45d-4509-914f-71d1d842bd9c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:10.833079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.834101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.834587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.835059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.837872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:10.838833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:10.838974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:10.839239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:10.851111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.860260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:10.884230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:10.885350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:10.886287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 332/691] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:10.897455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:10.917137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:10.918999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.038049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 360/692] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:11.044065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.059947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341c00-8935-4e63-b550-9a6bf08cb04d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.107444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.109003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.109745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.110202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.111779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:11.112566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.114746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a978b-d3a0-4cb8-88e1-7d9f48425c29 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.140952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 333/693] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:11.147001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2874c1-2fdf-44b1-a33f-8e3800eec05b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.147362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2874c1-2fdf-44b1-a33f-8e3800eec05b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.148187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2874c1-2fdf-44b1-a33f-8e3800eec05b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.149622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 334/694] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:11.150359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2874c1-2fdf-44b1-a33f-8e3800eec05b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.164980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dd5de-11de-43cf-97d2-fd1c7462f463 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.203226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.204359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.204848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.205393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.216169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.217887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3a1d0035-3997-449b-8b9c-f9d3e43cda47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:11.235347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:11.236298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:11.237214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:11.237214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:11.237214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.237214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:11.244380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 361/695] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:11.246003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.246503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.246969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.255303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2874c1-2fdf-44b1-a33f-8e3800eec05b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.264874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:11.266328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:11.292705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.293402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.293684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.293959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.335637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 362/696] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:11.337011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.364139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.365587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.434243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d54fd3-236c-4445-96cd-8b77265cdea4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.466612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 335/697] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:11.467480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.469866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.470791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.471264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.471742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.472685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:11.473211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.474779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.487293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 363/698] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:11.492957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.570245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f137-a546-4fef-9aa6-1e469e125203 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.590642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91ed4-9a19-4113-acad-545e88cedb28 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.604092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.604943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.605228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.605506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.616080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.617578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.626928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.628399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.628875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.629359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:11.635856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:11.636741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:11.637715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:11.637715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:11.637715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.637715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:11.637715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 336/699] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:11.638432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.643332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.645021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:11.662229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:11.663158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:11.664043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:11.664043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:11.664043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:11.664043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:11.666433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 364/700] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:11.667444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.743805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13328b3d-f9db-4bf6-bce2-02de687d25e0 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.767703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf39a65-b2f3-4e15-a149-5840d0b7d741 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.779914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.780791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.781076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.781353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.789329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.790880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:11.805979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:11.806942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:11.807823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:11.811760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.812239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 337/701] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:11.813036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.813534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.814111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.814573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.815495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:11.816000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.817606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.830677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 365/702] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:11.831815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.912786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61b8d5-c38b-41f9-9c7b-340e014dc0a1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.929031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804e80-00a9-47b9-8fcc-c84173ca4e8e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:11.946749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.947421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.947706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.948008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.948752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:11.949061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.950445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.965130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:11.966132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:11.966607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:11.967276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 338/703] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:11.967514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:11.968156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:11.968994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:11.969496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:11.971131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:11.983326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 366/704] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:11.984258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.068863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38d2d-3cab-40f1-a9d9-dce4a6dc1da6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.084901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4db4-85b0-4c63-a681-b3b07047ea1f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.102787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.103457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.103744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.104020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.104756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.106200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c652370-7150-4428-bf5e-c6efa7717e81 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.117190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.118292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.118597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.118917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.126050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.136287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.136498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 339/705] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:40:11 2022] POST /identity/v3/users => generated 254 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:12.138156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe9228-82c8-46a0-b5cc-f5a7d39c7f8c None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.138494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe9228-82c8-46a0-b5cc-f5a7d39c7f8c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.138777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe9228-82c8-46a0-b5cc-f5a7d39c7f8c None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.140672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 340/706] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:12.142226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe9228-82c8-46a0-b5cc-f5a7d39c7f8c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-99207cc3-def6-4fb5-9aaa-279d58508de8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:12.152405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:12.153299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:12.154181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 367/707] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:12.156086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.156086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.156086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.175559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:12.176756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:12.256783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 368/708] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:12.257801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.279966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe9228-82c8-46a0-b5cc-f5a7d39c7f8c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.325815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.325815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.325815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.326494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.328829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=86ce304b33114f83a8fa63a0227aeef5) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:12.328829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.329135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.337793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 341/709] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/role_assignments?user.id=86ce304b33114f83a8fa63a0227aeef5&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:12.338648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.382994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5feec0-e8a3-419b-8b42-2b0e09f0d247 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.421991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.423101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.423727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.424199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.425237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:12.425818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.427528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.441937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 369/710] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:12.448482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.470893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4579d-a587-4a57-b651-b8d8e1d2fe7a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.514721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.515611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.515842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.516115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] PATH_INFO: `/v3/services/network` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.517409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.519501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:12.534266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:12.535420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:12.536303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 342/711] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:12.542526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.566033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1747227-b309-4bf6-a41e-ff3351567a99 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.606967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.608901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.608901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.608901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.609155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:12.609417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.610884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.618249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 370/712] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:12.676975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c7abe-0b6b-45a9-bbd9-12dbe9467dd4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.715748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.716472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.716755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.717038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.717779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] RBAC: Adding query filter params (name=network) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:12.718091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.719711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4303328-8eb7-4e9d-b0d5-0d1f0e43de21 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.725199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 343/713] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:12.732027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:12.843571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b1168-30a5-48ba-b91c-5adad0dd9259 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:12.865973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdce4-d412-4880-9a5e-44db7d791f04 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.866488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdce4-d412-4880-9a5e-44db7d791f04 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.866588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdce4-d412-4880-9a5e-44db7d791f04 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.867931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 344/714] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:12.873450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.873733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.873996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.884957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:12.885932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:12.886419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:12.887230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:12.888129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] RBAC: Adding query filter params (type=network) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:12.888702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:12.890912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:12.895426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:12.896643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:12.898374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 371/715] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:12.971641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 345/716] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:40:12.994359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:13.128297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b77f77-9f9a-41fb-bffd-5af367b98392 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:13.176020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:13.176881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:13.177149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:13.177425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:13.178077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:13.179449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:13.198633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:13.199623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:13.200497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 372/717] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 12 00:40:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:13.206327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:13.320396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f03f2-b9da-45f1-a7cf-a66d1a21575f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:13.364686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:13.365360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:13.365638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:13.365906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:13.366620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:13.366911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:13.368427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dfb19-f050-4f3e-b40d-a31e0cda4c9d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:13.373099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 346/718] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:13.378949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:13.500589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616f59b-8c91-4fce-9d93-beb20d6efdfc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:13.533164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:13.533824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:13.534102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:13.534374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:13.535085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:13.535383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:13.536791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee78738-6af2-48ac-91fd-8020269c2c97 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:13.541436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 373/719] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:15.020329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15640ed-c9c0-46bc-b702-f2419e577d6a None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.020870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15640ed-c9c0-46bc-b702-f2419e577d6a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.021157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15640ed-c9c0-46bc-b702-f2419e577d6a None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.022944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 347/720] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:15.028888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.029311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.029491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.053295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:15.054510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:15.156399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 374/721] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 00:40:15.175665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15640ed-c9c0-46bc-b702-f2419e577d6a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.280396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15640ed-c9c0-46bc-b702-f2419e577d6a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:15.317390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:15.318197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.318475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.318746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.329344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6c577c08-de94-414b-a18f-41807cbd6846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:15.337402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:15.338402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:15.339220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:15.343463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 348/722] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:15.344881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.409000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4795d-fc1c-4041-801c-9fc88276ed8e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.409338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4795d-fc1c-4041-801c-9fc88276ed8e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.409548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4795d-fc1c-4041-801c-9fc88276ed8e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.410945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 349/723] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:15.416700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9915ebf-92e6-4b7c-bc7b-f91ccde876d7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.416965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9915ebf-92e6-4b7c-bc7b-f91ccde876d7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.417237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9915ebf-92e6-4b7c-bc7b-f91ccde876d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.445769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b9915ebf-92e6-4b7c-bc7b-f91ccde876d7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:15.446946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b9915ebf-92e6-4b7c-bc7b-f91ccde876d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:15.503094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0617d5-a7a6-429a-ab60-3b11b9ef1138 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:15.564828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 350/724] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:15.584226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:15.585123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d2181-a04f-4889-9996-5d69172719da None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.585397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.585509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d2181-a04f-4889-9996-5d69172719da None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.585673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d2181-a04f-4889-9996-5d69172719da None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.586283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.586725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.587533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 351/725] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:15.587781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:15.588230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:15.589832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:15.592528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.592814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.593086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.606764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 375/726] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:15.607623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.611245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:15.612616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:15.696994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 352/727] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:15.696994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.720964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fc684-22e4-47d7-b54f-0ed5a29b03e4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:15.758882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:15.759791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.760070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.760368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.770004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-53f0cfc7-a44c-461a-9a4e-d8025f8b6ce0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:15.777704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:15.778617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:15.779474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 376/728] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:15.780645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68ea7-b750-409c-b963-96b106ba9135 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.780645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68ea7-b750-409c-b963-96b106ba9135 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.780797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68ea7-b750-409c-b963-96b106ba9135 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.782304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 377/729] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:15.783030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f68ea7-b750-409c-b963-96b106ba9135 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.803832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368a76d-8b2f-4799-9588-718fe65eeb3c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:15.838841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:15.839558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.839826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.840122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.850268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:15.851896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:15.872754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:15.873723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 353/730] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:15.879585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:15.887137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f68ea7-b750-409c-b963-96b106ba9135 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:15.919464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:15.920298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.920571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.920841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.931543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-dc868f1d-080e-4887-98a0-7d481b82c29c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:15.939134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:15.940047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:15.940932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 378/731] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:15.941603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:15.942151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:15.942151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:15.959712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:15.960914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:15.988462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05234d1-50a9-49cd-a9b2-2eac41ac746d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.023656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.024536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.024619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.024897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.025650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.025957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.027565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.036037 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 354/732] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.042575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.042723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 379/733] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:16.042723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.136611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d96a9-a554-4576-95f5-2d10c5f04cf1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.139073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac64-c1d6-4f45-9627-040a43f654f7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.169399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.170160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.170501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.170846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.171598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.171903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.173299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.173622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.174344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.174671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.175678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.175984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.177414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.186151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 380/734] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.186151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.189469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 355/735] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.190490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.290467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244dbd7-cbfa-47aa-9471-8d6c2334e7dc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.292633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35406003-bc7d-4538-8e95-924e8aa4f71c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.323770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.324461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.324741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.325102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.325578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.325816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.326388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.326663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.326935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.327194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.339921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.341475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.344435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 356/736] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:16 2022] POST /identity/v3/services => generated 240 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:16.345291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:16.370450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:16.371449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:16.372426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:16.372426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:16.372426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:16.372426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:16.374983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 381/737] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:16.376221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.467102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e4-cb38-47f8-a141-fc7b37c654e7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.486005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dea5e-e37e-48a9-b6d9-4707311c96c7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.505872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196e1be-3866-4893-982f-e519acec682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.506739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196e1be-3866-4893-982f-e519acec682c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.507019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196e1be-3866-4893-982f-e519acec682c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.507286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196e1be-3866-4893-982f-e519acec682c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.518828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196e1be-3866-4893-982f-e519acec682c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.520332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196e1be-3866-4893-982f-e519acec682c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.520586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.521418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.521699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.521971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.530264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.531714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-0196e1be-3866-4893-982f-e519acec682c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:16.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:16.539832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:16.541246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:16.541246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:16.541246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:16.541246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:16.544401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 357/738] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:16.545636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196e1be-3866-4893-982f-e519acec682c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-7556d77b-cd66-493c-b151-80f537976e6e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:16.550704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:16.551669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:16.552670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 382/739] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:16.553246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82022fe5-05d0-40d4-9741-4f0ca7be231e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.553787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82022fe5-05d0-40d4-9741-4f0ca7be231e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.553787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82022fe5-05d0-40d4-9741-4f0ca7be231e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.556401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 383/740] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:16.556401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82022fe5-05d0-40d4-9741-4f0ca7be231e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.654727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196e1be-3866-4893-982f-e519acec682c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.656653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82022fe5-05d0-40d4-9741-4f0ca7be231e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.686947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.687617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.687900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.687999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.688183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.688654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.688931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.689023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.689206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.689299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.689909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.690207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.690620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.691545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258303-2af9-4494-9539-4843c219f648 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.702443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 358/741] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.703527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.707958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 384/742] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.709392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.709683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.709961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.725979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:16.727207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:16.804394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 385/743] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:16 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 00:40:16.805349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.808370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3999d8-da93-44f8-98c4-22af1974a4f6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.841411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.842084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.842380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.842636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.843502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:16.843809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.845273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.866552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 359/744] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:16.866552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:16.910056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575ffdc-1cb7-46d4-89b1-299901b268f6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:16.944748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:16.945641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:16.945964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:16.946217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] PATH_INFO: `/v3/users/nova` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:16.953313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:16.954742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:16.965257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acbaf-403d-46b9-bf4c-9552b0acbcab None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.003206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.003945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.004230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.004544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.013041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.014532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:17.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:17.031929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:17.032913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:17.036757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 360/745] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:17.037624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:17.094907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:17.096357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:17.097324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:17.100814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 386/746] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:17.101880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.145383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad903a-fc66-4ee6-89af-54785e963161 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.208316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95d5d-4f46-4c31-9460-a7b9d8ee505a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.245931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.246600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.246875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.247143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.307378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.307946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.308232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.308519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.309239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.310814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.315321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.316996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.336304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 361/747] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:17 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:17.336304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.446189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 387/748] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 00:40:17.447212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.462191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee2ecf-c42f-43ba-aba7-88e143ec9682 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.510407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.511414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.512171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.512492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.513643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:17.513951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.516208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.542064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 362/749] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:17.543094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.564101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d43c-c60b-4d4d-9f26-57c13c028c64 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.602804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.603815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.604323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.604777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.605806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:17.606293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.608611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.629292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 388/750] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:17.630402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.662942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf49d8b-c62a-4115-933b-94689902cd6a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.704318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.705621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.705978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.706321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.718914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.720924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.736170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a6b-2f5d-4030-9491-bcc7589550f4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:17.739556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:17.740605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:17.741709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:17.741709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:17.741709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:17.741709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:17.742494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 363/751] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:17.747256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:17.782899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.784866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.785256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.785631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.857372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2092b-1f10-481e-ae9c-5f741a9b5cef None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:17.879529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.881915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.892074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:17.893094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:17.893527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:17.893976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:17.905914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:17.907578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:17.925772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:17.926742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:17.927619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:17.927619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:17.927619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:17.927619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:17.930131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 364/752] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:17.931312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.003182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 389/753] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:18.004105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.047284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d34752-feed-4815-bc57-f061e5421bcb None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.084379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.085310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.085744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.086205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.087184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.087639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.089269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.109801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 365/754] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.111036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.113740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35abd7-e572-423e-b073-9f5d3e09f8b9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.153520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.154468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.154468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.154749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.155480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.155787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.157307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.170261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 390/755] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.171314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.214520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d1a6-28a3-4249-872e-980f853f996e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.254321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.255867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.256406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.257216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.271250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.274002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.279960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7972d4-db37-49f5-a792-f4e9a48de1c3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:18.292786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:18.293729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:18.294613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:18.294613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:18.294613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:18.294613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:18.296986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 366/756] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:18.298036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.320206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.321804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.322160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.322546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.329650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.331170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:18.347060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:18.347980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:18.353984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 391/757] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:18.354919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.403482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17270cd2-871d-4ca1-a537-5ef52deb1cf3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.438321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.439530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.440018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.440539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.448790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.450671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.462915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f98c45-7712-43d7-b05f-64ae26d87b3a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:18.466201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:18.467103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.468007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:18.470545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 367/758] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:18.471593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.499756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.500542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.500825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.501100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.501880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.502185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.503974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.520952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 392/759] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.521854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.576394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f96d-9f4b-4628-b4c9-3df27c8beab1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.611625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.612737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.613245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.613733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.614732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.615249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.616944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.629299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11521d3-50ed-44b2-8e3b-f0bdf9906dbf None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.630177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 368/760] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.630894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.668348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.668981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.669559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.669559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.670316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.670625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.672055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56faff2-4e74-47f6-b63a-3918ee104463 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.685446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 393/761] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.686743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165dec-4197-418e-95e1-d8370045657d None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.687030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165dec-4197-418e-95e1-d8370045657d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.687305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165dec-4197-418e-95e1-d8370045657d None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.688798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 394/762] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:18.689566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165dec-4197-418e-95e1-d8370045657d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.735995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3e55-6cbf-4d31-8694-9242be181464 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.768766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.769564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.769845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.770118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.776690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.778141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.790073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165dec-4197-418e-95e1-d8370045657d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:18.792572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:18.793460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:18.794322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 369/763] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:18.795028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.828591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.829774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.830113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.830383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.831261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=7f17488343744b5cb36683c882d4bf21) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:18.831717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.833227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1813-a2ef-4429-bff3-2572eee102ff None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.838765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 395/764] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/role_assignments?user.id=7f17488343744b5cb36683c882d4bf21&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:18.846111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.846889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.847731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.874599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:18.876220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:18.910311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ec11-1f28-401b-a5e9-698351f6b84e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:18.947798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:18.948485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:18.948770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:18.949046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:18.951195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=736e16d1bc05411faea9c943f930acb1, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:18.952760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:18.968750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 396/765] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:18.970149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:18.970977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 370/766] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:18 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:19.005624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.103443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca488e-db58-4228-8073-c7457f58c395 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.151067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.152048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba9021-2b68-4d22-91d4-20669481cdcd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.153012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.153511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.153981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.154972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:19.155496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.157229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.176906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 397/767] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:19.182971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.190384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.191227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.191502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.191769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] PATH_INFO: `/v3/services/network` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.192477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.193882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:19.207088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:19.208114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:19.208972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:19.219077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 371/768] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:19.220061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.317882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26d200-afab-4674-a404-01ae166cc946 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.350664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e802-b2f4-4ddd-82cd-3f604ddac047 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.353463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.354137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.354416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.354686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.355518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=86ce304b33114f83a8fa63a0227aeef5) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:19.355834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.357289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.363316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 398/769] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/role_assignments?user.id=86ce304b33114f83a8fa63a0227aeef5&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:19.420404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.421099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.421435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.421710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.422457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] RBAC: Adding query filter params (name=network) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:19.422770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.424312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.432834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 372/770] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:19.438796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.580891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963191-a763-4c01-b188-f7c2bae9000b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.623951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.624744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.625026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.625312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.626046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] RBAC: Adding query filter params (type=network) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:19.626353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.628151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.634677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 399/771] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:19.640798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.760342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930111c6-d995-4dd6-8690-8aa0905f07dd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:19.799756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:19.800457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.800744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.801023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.801837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] RBAC: Adding query filter params (interface=public, service_id=d9d738c82ab7438dae75cc442b68a02b, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:19.802152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:19.803543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a89ec-6b5a-4258-8eed-6a635216ae44 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:19.811952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 373/772] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/endpoints?service_id=d9d738c82ab7438dae75cc442b68a02b&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:19.817017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:19.917812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2cfb8-c70a-4b03-81ca-6d6531a4e311 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.918160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2cfb8-c70a-4b03-81ca-6d6531a4e311 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.918378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2cfb8-c70a-4b03-81ca-6d6531a4e311 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.920054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 374/773] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:19.926181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:19.926498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:19.926755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:19.961871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:19.963073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:19.982374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ed395-47c4-4b13-92a8-6001520a1b20 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:20.020392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:20.021069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:20.021351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:20.021626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:20.022313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:20.023776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:20.029559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 400/774] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:19 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:20.089765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 375/775] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:20.125318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:20.268163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929208-9bbf-4c27-9988-0d4c95ea3042 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:20.314416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:20.315258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:20.315532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:20.315810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:20.316503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:20.317869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:20.334273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:20.335247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:20.337578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 401/776] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 12 00:40:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:20.345389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:20.460072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9eddd-32e3-4af8-9aaa-c7f069cb9bd8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:20.494613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:20.495299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:20.495582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:20.495856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:20.496627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:20.496936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:20.498369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:20.504603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 376/777] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:20.510011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:20.655404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542222e-82e8-4cde-8506-ea8ad43949d1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:20.698749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:20.699422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:20.699705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:20.699978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:20.700729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:20.701037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:20.702407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:20.711175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 402/778] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:20.717161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:20.823786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86addd34-5a5e-42a6-b178-8d5678ea2354 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:20.864082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:20.864801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:20.865085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:20.865360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:20.866196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] RBAC: Adding query filter params (interface=public, service_id=ac6d2714b8dc414d914e4184ea92d17c, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:20.866509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:20.868409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e761e-86a9-4bdc-891c-df849dd60ba6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:20.874420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 377/779] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:20 2022] GET /identity/v3/endpoints?service_id=ac6d2714b8dc414d914e4184ea92d17c&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:20.880459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:21.026081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5136d6-3698-450c-9e9e-663672d34fd3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:21.077074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:21.077765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.078052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.078327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.079026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:21.080431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfc0db-c3ab-450b-b69e-64b5a744b2d8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:21.086447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 403/780] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:20 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:21.539477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190087-762d-4ffe-af34-34f720f5f916 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.540062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190087-762d-4ffe-af34-34f720f5f916 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.540062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190087-762d-4ffe-af34-34f720f5f916 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.541646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 378/781] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:21.547167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58176bd1-53a8-4380-a65a-2c4405fac315 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.547582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58176bd1-53a8-4380-a65a-2c4405fac315 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.547759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58176bd1-53a8-4380-a65a-2c4405fac315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.573430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-58176bd1-53a8-4380-a65a-2c4405fac315 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:21.574643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-58176bd1-53a8-4380-a65a-2c4405fac315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:21.662670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 404/782] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:21.697688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190087-762d-4ffe-af34-34f720f5f916 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:21.853199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190087-762d-4ffe-af34-34f720f5f916 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:21.868577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2fe64-66b4-41cc-ae4d-961e4ddc6c04 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.868925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2fe64-66b4-41cc-ae4d-961e4ddc6c04 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.869120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2fe64-66b4-41cc-ae4d-961e4ddc6c04 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.870772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 405/783] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:21.875945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.876229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.876533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.903949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:21.905468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:21.921476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:21.922638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:21.923101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:21.923570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:21.948777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:21.960500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:21.961843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:21.962681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:21.965115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 379/784] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:21.972428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.053792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 406/785] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:22.093034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.109078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51de8-b3fd-4a88-b186-2c461505e89e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.175605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.176297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.176587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.176865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.177625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:22.177932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.179318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.192437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 380/786] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:22.199576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.284541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45dc12-ca8a-4c92-a431-d1d60b743cf6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.324988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.325799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.326083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.326364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.336841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-734f4382-63e1-4aea-bb69-439369700fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:22.344720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:22.345679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:22.346499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 407/787] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:22.347675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e00e-6cfa-4ea1-a0af-b160fd7df003 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.347834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e00e-6cfa-4ea1-a0af-b160fd7df003 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.348102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e00e-6cfa-4ea1-a0af-b160fd7df003 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.349211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6c84-09b9-488f-8f88-854da1fd3f21 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.354452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 408/788] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:22.358792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e00e-6cfa-4ea1-a0af-b160fd7df003 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.396916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.398013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.398355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.398636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.411714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.414110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b8b194bf-a183-4964-98a2-d7e6015afb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:22.435204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:22.436167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:22.437177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:22.437177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:22.437177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:22.437177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:22.440320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 381/789] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:22.441798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.442086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.442366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.463359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:22.465383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:22.481911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e00e-6cfa-4ea1-a0af-b160fd7df003 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.522143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.522854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.523339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.523631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.524413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:22.524715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.526773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.541864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 409/790] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:22.541864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.553364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 382/791] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:22.554965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.678289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25edbeb-79a4-4630-a144-80447f89e682 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.683693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a10d3-081b-44af-8a8e-90f8887e3590 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.716761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.717434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.717716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.717988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.718720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:22.719015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.719140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.719926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.720298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.720423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.720685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.731140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.732662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 410/792] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:22.732790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.733571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:22.752119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:22.753556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:22.754448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:22.754448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:22.754448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:22.754448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:22.754448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 383/793] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:22.756026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.832844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e88a0-4d76-483b-b63b-4890b142747d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.850946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b08d-5a7c-4990-905a-c9339f30e275 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:22.865977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.866781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.867134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.867360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.877444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:22.883780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:22.884629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.884910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.885182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:22.885847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:22.886903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:22.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 411/794] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:22.888798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.898730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:22.900185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-bb8845d6-0a06-4571-b5ae-bf8facc30766 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:22.915731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:22.916720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:22.917642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 384/795] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:22.918733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d319ec2-8fc9-4e8e-9321-0c0be8f66109 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:22.918733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d319ec2-8fc9-4e8e-9321-0c0be8f66109 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:22.918733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d319ec2-8fc9-4e8e-9321-0c0be8f66109 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:22.924284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 385/796] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:22.924284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d319ec2-8fc9-4e8e-9321-0c0be8f66109 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:22.995648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27926b-6ff0-4ade-910a-6aad3198d62d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.019968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d319ec2-8fc9-4e8e-9321-0c0be8f66109 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.029179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.030016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.030465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.030885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.031741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.032185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.033707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.050358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 412/797] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.051490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.053951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.055042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.055467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.055744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.056548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.056854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.058238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde3ef-0374-4d7d-af2e-3d8f2b86987e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.072500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 386/798] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.072500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.072500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.072500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.085865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:23.087055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:23.155004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 387/799] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:23.155866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.172320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e053-b72f-4710-8b61-96a4fb226792 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.206765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.207442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.207726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.207999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.208871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.209181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.210602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.227725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 413/800] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.228615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.258601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcc3c9-a966-4e5e-aaf8-1388342a3a60 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.291554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.292387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.292673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.292948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.299940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.301432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:23.316167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:23.317289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 388/801] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:23.318213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.331731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc410-d0b8-472d-a75b-887a44a8376c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.364201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.365029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.365311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.365588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.375604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.377082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:23.394485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:23.395472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:23.396404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:23.396404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:23.396404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:23.396404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:23.396404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 414/802] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:23.396684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.422862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87ce6b-abab-458c-a4b8-07b5b813cbbf None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.458470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.459275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.459557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.459833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] PATH_INFO: `/v3/services/network` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.460526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.462193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:23.473686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:23.474722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 00:40:23.475551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:23.479999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 389/803] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:23.481028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.497860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b2fa-28cd-4224-b669-d2af83e174bf None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.533382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.534144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.534426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.534702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.545572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.547032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:23.564566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:23.565537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:23.566425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:23.566425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:23.566425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:23.566425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:23.571071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 415/804] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:23.572090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.586551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe742-0e40-4cfc-aae3-9dbdebbe8374 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.621476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.622164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.622450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.622726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.623576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.623891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.625370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.643005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 390/805] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.643818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.685367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c2ee-ccbd-48a0-8a29-84eedd553cfd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.719494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.719950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.720232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.720535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.721263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.721684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.722939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.734362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 416/806] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.735414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.744782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5746f-1c82-46e2-b66d-982e592589d6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.778975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.779643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.779925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.780200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.781044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] RBAC: Adding query filter params (name=network) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.781357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.782722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.787057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 391/807] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.787812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.831440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac45c-6c34-4513-930f-59a4cb2b180d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.863429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.864090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.864404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.864679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.865406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:23.865715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.867269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:23.879857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 417/808] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:23.880688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:23.887396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc63c90-88b9-42d1-a285-1206645bb3df None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.920356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:23.921011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:23.921300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:23.921583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:23.980812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd3646-304a-458c-8dee-7c731f54c1e9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:23.984584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:23.986163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.013949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.014741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.015019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.015295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.022139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.023694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-f9c80e1b-b235-470f-afde-2d75c3f6f402 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:24.037693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:24.038661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:24.040304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 418/809] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:24.041051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc77a4-2aec-42ff-ab5f-9ab297580d65 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.041051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc77a4-2aec-42ff-ab5f-9ab297580d65 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.041051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc77a4-2aec-42ff-ab5f-9ab297580d65 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.042133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 419/810] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:24.042819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc77a4-2aec-42ff-ab5f-9ab297580d65 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.090696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 392/811] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:24.091893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.140890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc77a4-2aec-42ff-ab5f-9ab297580d65 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.174073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.174734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.174991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.175297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.176002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] RBAC: Adding query filter params (type=network) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:24.176343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.177704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.182615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 420/812] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:24.183390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.187760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9566-36a9-4875-9e69-dead71d294c3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.221442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.222225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.222498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.222765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.229321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.230756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-3319199c-73d6-4305-82da-6f45b9f91997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:24.244000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:24.245113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:24.246058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 393/813] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:24.246533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.246609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.246948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.263710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:24.264905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:24.281142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d08cb-6e3a-44f0-82f9-705b870deefa None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.313947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.314535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.314823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.315098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.315931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:24.316241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.317680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.329655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 394/814] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:24.330491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.343998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 421/815] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:24.344671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.423244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e35dd-f5af-48aa-8042-950cb6533c89 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.439175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbe110-7c6a-4bbb-831d-ebe4812dd92c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.456618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.457415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.457699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.457983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.468050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.469541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.471178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.471828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.472107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.472407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.473139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.474481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:24.487874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:24.488986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 395/816] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:24.490075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.497864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 422/817] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:24.498677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.598533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268141d-81b8-46da-8396-cf24b08f2dc6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.610640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78577f-6f1c-4fd7-b294-132c42ffdc5e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.639291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.640127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.640555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.640920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.641798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:24.642166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.643889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.649279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.650216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.650496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.650767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.657457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 396/818] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:24.658665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.663221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.664206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:24.682411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:24.683488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 423/819] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:24.684776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.761277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3532-06ca-4dea-aa28-238f63f094e0 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.785059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeead-9d83-4d4a-8640-20dfb68cc7fa None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.792426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.793232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.793478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.793742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.794392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.795762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:24.807361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:24.808342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 00:40:24.809174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:24.810075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 397/820] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:24.810690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.817826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.818493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.818806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.819074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.819798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:24.820218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.821737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.833460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 424/821] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:24.834248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.911695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf76a0-a3f4-49c2-ad25-2444f3ebd583 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.932110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a58a-8fc0-463e-83ab-fad7db08ffe2 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:24.950099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.951524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.951933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.952383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/7f17488343744b5cb36683c882d4bf21/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.967245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:24.967907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:24.968182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:24.968491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:24.969210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:24.969514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.970878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.973296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=7f17488343744b5cb36683c882d4bf21, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:24.974735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:24.982303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 425/822] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:24.983263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:24.991489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 398/823] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:24 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/7f17488343744b5cb36683c882d4bf21/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:24.992700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.088230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5e7f-c111-4233-ba06-7ea006cadf27 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.107604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77fe23-b2ed-4cf4-897c-bda00c283b0d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.124498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.125671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.125671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.126379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.127232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:25.128016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.129596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.140331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 426/824] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:25.140331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.147455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.148096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.148410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.149324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.157045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.158552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:25.174953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:25.176042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:25.177148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:25.180417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 399/825] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:25.181508 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.250271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627f58-656a-46c2-b43f-c82fe751453a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.282498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec99b-85d2-4c51-a0ef-e03a8ef8aa10 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.283949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.284783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.285067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.285386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.292798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.294248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:25.308963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:25.309855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:25.310737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 427/826] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:25.311741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.319971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.320721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.321000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.321276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.322257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:25.322754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.324718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.331053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 400/827] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 12 00:40:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:25.331677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.413412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec989a3-c124-4126-bbec-a62435fb72b5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.441043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b998c69-0e73-49c5-817d-6ca1417dd8a6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.448003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.448693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.448976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.449250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.450042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:25.450348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.451768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.464104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 428/828] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:25.465305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.478897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.479679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.480026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.480403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.481207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:25.481518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.482956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.500684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 401/829] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:25.501583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.566959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59389f2b-15fe-43ae-bc37-87aa3e854659 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.598690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c136c-e47a-4598-84c2-c90eca916828 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.600559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.601453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.601843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.602179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.602988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.604474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.624916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 429/830] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:25.625573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:25.651582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.652690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.653211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.653731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.654794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:25.655316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.657353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744cb8-913b-4825-b49d-334d5999998e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.665164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 402/831] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:25 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:25.738052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8868-05d4-4a04-8538-466bf65fa425 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:25.777795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:25.778710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:25.778987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:25.779256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/86ce304b33114f83a8fa63a0227aeef5/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:25.781436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=86ce304b33114f83a8fa63a0227aeef5, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:25.783006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c99299-dfc8-49d2-ae7d-6902906051bd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:25.801515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 430/832] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:25 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/86ce304b33114f83a8fa63a0227aeef5/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:27.339584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c29c-679e-4805-aef7-8a62fc726cd8 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.340480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c29c-679e-4805-aef7-8a62fc726cd8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.340905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c29c-679e-4805-aef7-8a62fc726cd8 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.342654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 403/833] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:27.348402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.349255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.349742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.367339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:27.368924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:27.436529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 431/834] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:27.460351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8c29c-679e-4805-aef7-8a62fc726cd8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:27.565046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8c29c-679e-4805-aef7-8a62fc726cd8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:27.600310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:27.601678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.602205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.602683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.613692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:27.621750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:27.622693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:40:27.623541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:27.626270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 404/835] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:27.632773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:27.731275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17f6db-daa5-46bb-81da-8b9c6ed7d76f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:27.766804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:27.768090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.768655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.769148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.770184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:27.770749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:27.772526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbc172-f682-4fb4-aa8e-e457daebd44c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:27.783436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 432/836] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:40:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:27.789894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:27.898686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e077f-6366-4c4e-b130-e66fd51df961 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:27.934082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:27.934898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.935180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.935451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.946517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:27.948310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-5b34acf6-2031-41c1-a5b7-1cf4894c61a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:27.966105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:27.967062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:27.967965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:27.967965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:27.967965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:27.967965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:27.968197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959a88-f4c0-4190-a4a8-20ce50458657 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.968197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959a88-f4c0-4190-a4a8-20ce50458657 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.968197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959a88-f4c0-4190-a4a8-20ce50458657 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.969407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 433/837] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:27.970831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 405/838] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:27.971412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406cad3-d48b-4a62-9cdc-c5b341d07b2f None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.971986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406cad3-d48b-4a62-9cdc-c5b341d07b2f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.972494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406cad3-d48b-4a62-9cdc-c5b341d07b2f None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.974234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 434/839] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:27.976028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406cad3-d48b-4a62-9cdc-c5b341d07b2f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:27.976592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb639-3113-4a01-b1d2-ca8e025f6c3c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:27.976920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb639-3113-4a01-b1d2-ca8e025f6c3c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:27.977152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb639-3113-4a01-b1d2-ca8e025f6c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:27.993727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-497fb639-3113-4a01-b1d2-ca8e025f6c3c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:27.994833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-497fb639-3113-4a01-b1d2-ca8e025f6c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:28.059113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 406/840] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:28.060441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.060723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.060998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.076773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406cad3-d48b-4a62-9cdc-c5b341d07b2f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.077326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:28.078581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:28.110489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.111138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.111416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.111687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.112431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.112739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.114092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6b8bb-c2b9-423d-80af-0a469a05c16b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.126288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 435/841] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.128171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3751-c229-4733-99a5-42383e98f58f None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.128489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3751-c229-4733-99a5-42383e98f58f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.128901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3751-c229-4733-99a5-42383e98f58f None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.130560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 436/842] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:28.131295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3751-c229-4733-99a5-42383e98f58f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.146711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 407/843] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:28.147948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.226320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3751-c229-4733-99a5-42383e98f58f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.243455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046b838-dfbc-48ff-a9ad-c105eaa1a5d8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.259364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.260088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.260402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.260669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.261319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.262666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3c204fda-829c-4b82-a4ac-0df1e00792b8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:28.274251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:28.275285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:28.281201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 437/844] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:28.281987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.282075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.282075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.282204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.282273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.282339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.282339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.285376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.286830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.293538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:28.294723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:28.301362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:28.302317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:28.303227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 408/845] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:28.303810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.356980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 438/846] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:28.357804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.403138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c58b1-22b1-4aab-a59f-9e5844bf0b8f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.434504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.435322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.435603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.435876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.436562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.437980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:28.450605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:28.451577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:28.452705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:28.453516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b35e63-9bf6-4c4c-bbf0-9d7126dfc721 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.453599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 409/847] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:28.453599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.484886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.485537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.485815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.486082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.486776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.487078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.488458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.493047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 439/848] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.493806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.549769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d67da-fce1-4a54-ab82-25b417b6668f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.582720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.583387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.583669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.583943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.584836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.585145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.586762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.590619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5c80b-794c-46d9-a57c-2c4cd1ad348e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.603237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 410/849] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.604045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.625264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.626062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.626338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.626608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.637255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:28.644781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:28.645750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:28.646566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 440/850] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:28.647917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.696567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdb225-bd18-474a-91d4-52e658c5ed27 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.729771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.730427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.730725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.731045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.731759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.732052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.733469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.737821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 411/851] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.738593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.746585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2dfa-907c-402e-9b2a-be2622a79b40 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.778539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.779201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.779478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.779747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.780472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.780795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.782149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.796389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 441/852] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.796389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:28.843993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c3e8-791c-49fe-865d-5785f2be2b4f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.879038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.879692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.879973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.880277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.901955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c23f6-04d2-41b7-ab43-d21b596494e0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:28.938092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:28.940143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:28.940684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:28.941313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:28.942890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:28.943429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.946250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.951275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:28.952798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:28.962840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 442/853] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:28.962840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.057381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 412/854] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:29.058167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dd75d-8056-4105-8f0f-a198ed146469 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.058285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.097985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.098574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.098853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.099865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.099865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:29.100122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.101630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.106704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 443/855] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:29.117496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.179674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00a727-2ed3-4b17-9f65-dde8ef82c401 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.223929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.225743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.226334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.227814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.242108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.245414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.246778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a74df1-f334-4568-a441-3fb5f6f1330a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:29.271866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:29.272856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:29.274599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:29.274599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:29.274599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:29.274599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:29.277461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 413/856] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:29.283778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.288013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.288861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.289272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.289649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.301351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.302975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:29.322919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:29.323895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:29.324881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:29.324881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:29.324881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:29.324881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:29.324881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 444/857] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:29.330764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.401384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd34ca-0dee-44e6-8917-603f960b1f5c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.436827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.437523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.437751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.440273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.440273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:29.440273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.440611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.452783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad018a0-a5c0-4aaa-a99e-2b9cca82ecbb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.456733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 414/858] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:29.457598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.487473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.488303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.488663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.489017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.489818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:29.490192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.491630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.508920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 445/859] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:29.510027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.558743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b23f1-1b84-449a-8125-9319c5f55d81 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.592397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.593201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.593474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.593745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.601745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.603193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.609918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125331db-9f53-4991-bf35-1c9bb689bd7d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:29.617866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:29.620130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 00:40:29.622381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:29.628716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 415/860] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:29.628716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.645919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.646752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.647076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.647347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.654734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.656115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:29.670385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:29.672587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:29.674417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 446/861] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:29.681431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.737725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c7de-ede1-4106-a3c0-c84baa1061d4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.772593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.773885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.778674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.779414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.780803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:29.781364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.783240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.783452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bf237-d4e5-48fb-9cb4-c738e1440a81 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.802492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 416/862] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:29.808242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.824726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.825400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.825682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.825962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.826902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:29.827229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:29.829079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:29.848331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 447/863] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:29.850345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:29.923744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549d95-370e-429e-b0cd-cc37d6accb83 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.957617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.958303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.958580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.958855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.959563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e552-6446-4361-841b-13d54fe787ab None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:29.996865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:29.997947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:29.998455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:29.998934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:29.999974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=7f17488343744b5cb36683c882d4bf21) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:30.000524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.002167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.016737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 448/864] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/role_assignments?user.id=7f17488343744b5cb36683c882d4bf21&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:30.030129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.031633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.138520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 417/865] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 12 00:40:30.145494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:30.251640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276f45-0eb7-403d-92f8-b057e306dcda None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:30.286475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:30.287606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:30.288057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:30.288551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:30.298999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.300756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:30.318438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:30.319433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:30.320559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:30.320559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:30.320559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:30.320559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:30.321836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 449/866] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:30.327596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:30.425442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cef1e-e507-402a-b426-336b7f1849ac None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:30.459160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:30.460732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:30.461204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:30.461695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:30.462593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:30.463073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.464661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.476633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 418/867] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:30.488279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:30.584704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5ff55-fd89-49d2-82fa-0593397e7e6d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:30.618877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:30.619693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:30.619977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:30.620269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:30.627734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.629213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:30.643003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:30.643943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:30.644879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 450/868] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:30.650445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:30.753217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ba6b-d289-4bf7-bf59-48b3dd24a7a8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:30.788691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b24d1-061a-4465-af54-f70138574cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:30.789874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b24d1-061a-4465-af54-f70138574cda None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:30.790364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b24d1-061a-4465-af54-f70138574cda None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:30.790839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b24d1-061a-4465-af54-f70138574cda None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:30.791826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b24d1-061a-4465-af54-f70138574cda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:30.792357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b24d1-061a-4465-af54-f70138574cda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.794009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b24d1-061a-4465-af54-f70138574cda None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.806710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 419/869] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:30.814042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:30.914985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb981b-d381-49f6-96cb-4b8477e0a58f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:30.952058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:30.952776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:30.953061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:30.953335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:30.954190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=86ce304b33114f83a8fa63a0227aeef5) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:30.954512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:30.956019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d588-d78b-4049-b901-012b0d609903 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:30.963462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 451/870] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:30 2022] GET /identity/v3/role_assignments?user.id=86ce304b33114f83a8fa63a0227aeef5&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:31.139030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4ec8-c6af-483d-a316-07f37106b1a7 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.139389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4ec8-c6af-483d-a316-07f37106b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.139589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f4ec8-c6af-483d-a316-07f37106b1a7 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.141295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 420/871] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:31.147686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9901a-3286-4245-a721-e31a8313ee6d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.147906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9901a-3286-4245-a721-e31a8313ee6d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.148301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9901a-3286-4245-a721-e31a8313ee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.169717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b0c9901a-3286-4245-a721-e31a8313ee6d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:31.171274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b0c9901a-3286-4245-a721-e31a8313ee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:31.252783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 452/872] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:31.272637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f4ec8-c6af-483d-a316-07f37106b1a7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:31.377283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f4ec8-c6af-483d-a316-07f37106b1a7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:31.393585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e3d0-3676-4eeb-a472-dfaba11cc395 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.394191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e3d0-3676-4eeb-a472-dfaba11cc395 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.394654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950e3d0-3676-4eeb-a472-dfaba11cc395 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.396196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 453/873] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:31.402489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4554d-a946-4122-aa62-38e2239311e2 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.403009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4554d-a946-4122-aa62-38e2239311e2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.403570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4554d-a946-4122-aa62-38e2239311e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.411432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:31.412100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.412412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.412688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.413417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:31.414908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:31.422670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-20c4554d-a946-4122-aa62-38e2239311e2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:31.423860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-20c4554d-a946-4122-aa62-38e2239311e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:31.434253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 421/874] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:31 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:31.521399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 454/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:31.543639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:31.656032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023874-fdc9-43ec-b861-5c78795d806e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:31.688236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:31.688860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:31.689139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:31.689409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:31.690136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:31.691504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccc0d9-3e66-4bc9-99f9-7022e9ef5e1c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:31.707600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 422/876] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:31 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:32.514456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251625-b70c-411b-9716-7876b52d2cd6 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:32.514638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251625-b70c-411b-9716-7876b52d2cd6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:32.514983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251625-b70c-411b-9716-7876b52d2cd6 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:32.516991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 455/877] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:32.522705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:32.523915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:32.524439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:32.542254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:32.543926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:32.628357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 423/878] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:32.645785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35251625-b70c-411b-9716-7876b52d2cd6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:32.748563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35251625-b70c-411b-9716-7876b52d2cd6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:32.786315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:32.788084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:32.788084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:32.788084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:32.798708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:32.807495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:32.808480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:32.809304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:32.811659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 456/879] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:40:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:32.817863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:32.922925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86627236-d263-45d5-a0b8-ef5940d329b7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:32.959340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:32.960026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:32.960339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:32.960836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:32.962152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:32.962465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:32.963911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:32.976550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 424/880] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:40:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:32.978774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.080935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abb99-2d68-4fe6-aee8-4b53c851415c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.113874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.115035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.115541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.116082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.127726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.129586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:33.151250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:33.152781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:33.153675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:33.153675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:33.153675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:33.153675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:33.153675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 457/881] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:33.157628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.262151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b70ce-4db5-4215-93f3-eb72d3968300 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.297548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.298619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.299118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.299623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.300608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:33.301111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.302696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3f617-316e-43ad-b156-0233d3f002e7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.316089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 425/882] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:33.322149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.331029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a56-5162-42e7-af82-43927f0e6783 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.331298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a56-5162-42e7-af82-43927f0e6783 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.331528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c3a56-5162-42e7-af82-43927f0e6783 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.333194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 426/883] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:33.339184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.339473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.339741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.357720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:33.358935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:33.429029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 427/884] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:33.449580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a012fbd-9f24-4b3f-a9d4-25680d6613ea None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.451403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.490636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.491972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.493113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.493614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.503991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.506195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:33.527428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:33.528477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:33.529412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:33.531360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 458/885] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:33.535992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.572759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db1bb-ab8a-47dc-9aea-347ad65fcaca None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.611063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.612311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.612799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.613242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.614157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.615825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:33.629421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:33.630610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:33.631474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:33.634640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 428/886] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:33.642212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.652107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502fce-98d3-485d-9c18-846f74ede5ac None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.686031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.687058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.687522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.687972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.689024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:33.689520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.691235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.713826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 459/887] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:33.715007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.742470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851e27-636c-47e5-9416-7188a3808a19 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.777785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.778526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.778811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.779083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.779824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:33.780127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.781619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.792860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 429/888] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:33.793950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:33.823513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cb05b-bc2b-4ba6-a078-e988d9f70dd9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.857786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.858444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.858725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.859004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.897253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5b46-2fdd-431c-a2f2-e6a55d8bd998 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:33.930485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.930941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:33.931617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.931897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.932168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.932909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:33.933213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:33.933405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.934575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eae7f-0980-4058-87d0-f175ae2c7c4d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:33.939118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 430/889] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:33.940484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa28c-666e-4d04-9a5b-277928ac6bb4 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.940771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa28c-666e-4d04-9a5b-277928ac6bb4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.941046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa28c-666e-4d04-9a5b-277928ac6bb4 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.947384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 431/890] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:33.949561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4ff0-e7e9-497a-bab0-d9ee9dd41714 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:33.949849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4ff0-e7e9-497a-bab0-d9ee9dd41714 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:33.950122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4ff0-e7e9-497a-bab0-d9ee9dd41714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:33.969735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-735c4ff0-e7e9-497a-bab0-d9ee9dd41714 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:33.971075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-735c4ff0-e7e9-497a-bab0-d9ee9dd41714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:34.044807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 432/891] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:34.046229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f677a-cb1c-414f-859f-684815a4ad7b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.046521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f677a-cb1c-414f-859f-684815a4ad7b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.046797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f677a-cb1c-414f-859f-684815a4ad7b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.048387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 433/892] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:34.054063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.054402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.054675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.070659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 460/893] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:34.072009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.072276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:34.073629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:34.148551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 434/894] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:34.149425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.186666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35759bd-699e-4714-85b3-c60c480f52a6 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.222282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.223096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.223373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.223645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.224333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.225702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:34.237415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:34.238419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.239227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 461/895] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:34.240024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.260962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696cd97-868c-49c1-b73a-6f7d3c657f5c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.294470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.295268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.295548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.295818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.305908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.307359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:34.324792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:34.325738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:34.327240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:34.327240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:34.327240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:34.327240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:34.327240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 435/896] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:34.327532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.343956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4881b-76f9-41a5-8c9f-96d438bb7172 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.376972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.378147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.378424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.378693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.379343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.380744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:34.393555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:34.394517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:34.395427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 462/897] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:34.396283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.430051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc861-5a4b-4a64-92bf-44708569a702 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.463728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.464437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.464719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.464994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.465710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.466016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.467396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.478685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 436/898] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.478685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.500603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eb6aa-3a31-4c9a-9fb9-5e89b80e49fc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.533517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.534186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.534466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.534739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.535459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.535762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.537154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.553358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 463/899] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.553358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.574462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4ea3-4177-4a8f-9606-b8b767fdc3f6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.609355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.610028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.610309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.610584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.611290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.611595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.612994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.618451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 437/900] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.619525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.649313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0b24-d491-45ba-8a0b-99a4819cef61 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.681495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.682126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.682352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.682623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.683319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.683618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.685068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.695032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 464/901] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.695725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.723804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba42ec-49fc-4595-bd86-6e929877fbea None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.758181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.758975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.759249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.759515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.766248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.767687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:34.781473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:34.782443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:34.783316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 438/902] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:34.783805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.794145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b7afa-3875-4cbb-b254-f5410d784c35 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.826802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.827460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.827737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.828011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.828743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.829041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.830402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.840017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 465/903] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.840017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.889465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76047a-ff1d-4469-b345-5e679d311597 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.921731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.922392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.922671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.922942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.923727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] RBAC: Adding query filter params (interface=public, service_id=cf2a52b948a3498bb82b136128a27689, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.924035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.925459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.930314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 439/904] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/endpoints?service_id=cf2a52b948a3498bb82b136128a27689&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.930314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:34.938083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520e5a-0389-4ab4-8d5b-9d9bf26fd5e9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:34.971216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:34.971874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:34.972152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:34.972448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:34.973226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:34.973532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:34.974953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:34.987586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 466/905] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:34.988656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:35.025921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce376-d781-489a-b39f-d2576d4b2624 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:35.060367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:35.061456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:35.061947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:35.062431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:35.063458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] RBAC: Adding query filter params (interface=public, service_id=ca806965c0e4472e83436eb894c7d0e4, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:35.064020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:35.065735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:35.075931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 440/906] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/endpoints?service_id=ca806965c0e4472e83436eb894c7d0e4&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:35.077105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:35.094568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2d553-d894-4f1f-8e4d-cb9686beed75 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:35.130181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:35.130863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:35.131145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:35.131422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:35.132101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:35.133722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:35.148850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 467/907] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:34 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:40:35.149764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:35.184972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab43b8-6bf5-4acc-8fd6-1a30e6cd9749 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:35.222416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:35.223035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:35.223316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:35.223592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:35.224503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] RBAC: Adding query filter params (role.id=d41561fab1b144bb982fce5f82ef571d, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=7f17488343744b5cb36683c882d4bf21) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:35.224817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:35.226524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646c93f-6649-469e-9752-c9fb823762af None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:35.233004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 441/908] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:35 2022] GET /identity/v3/role_assignments?user.id=7f17488343744b5cb36683c882d4bf21&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=d41561fab1b144bb982fce5f82ef571d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:35.287042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040351-43d1-4217-befe-8c0648761906 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:35.327024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:35.327725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:35.328006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:35.328313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:35.329029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:35.330459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2dc6-bf9d-4656-887b-facb6f8fa78c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:35.337907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 468/909] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:35 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:40:36.183516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17afec-2144-466f-9501-1d9a48a501d8 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:36.184547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17afec-2144-466f-9501-1d9a48a501d8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:36.185021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17afec-2144-466f-9501-1d9a48a501d8 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:36.186912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 442/910] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:36.192637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ad69-e1aa-48a5-b3c4-cd4a3c30e0c6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:36.193067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ad69-e1aa-48a5-b3c4-cd4a3c30e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:36.193241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193ad69-e1aa-48a5-b3c4-cd4a3c30e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:36.212345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8193ad69-e1aa-48a5-b3c4-cd4a3c30e0c6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:36.213535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8193ad69-e1aa-48a5-b3c4-cd4a3c30e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:36.279023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 469/911] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:36.301495 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17afec-2144-466f-9501-1d9a48a501d8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:36.406541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17afec-2144-466f-9501-1d9a48a501d8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:36.439971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:36.441055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:36.441532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:36.442017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:36.442953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:36.444543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefdbd4-be89-4b2b-bc7f-802dd18cd11d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:36.461114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 443/912] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:36 2022] POST /identity/v3/services => generated 241 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:37.430771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabdfce-e90b-4d1f-b237-c97299a62f52 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.431305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabdfce-e90b-4d1f-b237-c97299a62f52 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.431305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabdfce-e90b-4d1f-b237-c97299a62f52 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.433011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 470/913] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:37.438917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.439362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.439458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.456747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:37.457928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:37.481250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba17f4-9b11-4965-bd83-ffbcd4c42135 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.481559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba17f4-9b11-4965-bd83-ffbcd4c42135 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.481802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba17f4-9b11-4965-bd83-ffbcd4c42135 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.483384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 471/914] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:37.488444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f3f1-7c64-4b4d-b714-045f8357d4bc None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.488734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f3f1-7c64-4b4d-b714-045f8357d4bc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.489016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f3f1-7c64-4b4d-b714-045f8357d4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.505780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4023f3f1-7c64-4b4d-b714-045f8357d4bc None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:37.506968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4023f3f1-7c64-4b4d-b714-045f8357d4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:37.536331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 444/915] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:37.553796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:37.573663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 472/916] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:37.594408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5b33-0002-4ea0-8dd8-009d8f925e63 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.594744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5b33-0002-4ea0-8dd8-009d8f925e63 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.594949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a5b33-0002-4ea0-8dd8-009d8f925e63 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.596418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 473/917] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:37.597135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a5b33-0002-4ea0-8dd8-009d8f925e63 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:37.665532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ed2c-c938-4bf6-b12b-b2ac23246c70 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:37.699225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:37.700020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.700235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.700536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.701189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:37.701584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a5b33-0002-4ea0-8dd8-009d8f925e63 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:37.702565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a5b27284-0411-4f4e-ae51-1e84ac767f8c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:37.714914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:37.715916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:37.716782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 445/918] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:37.718123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.718123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.718123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.733965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:37.735128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:37.740228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:37.741080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.741361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.741714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.752709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:37.761139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:37.762101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:37.762940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 474/919] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:37.763632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:37.806588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 446/920] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:37.809594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:37.867210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27740908-3e2a-417a-9e09-8a9ee6028e98 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:37.899901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:37.900598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.900875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.901142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.901847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:37.902152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:37.903517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:37.908014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 475/921] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:37.908815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:37.924935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de669cf7-2d61-495f-a2f7-a51114cf7165 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:37.962278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:37.962865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:37.963142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:37.963397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:37.964158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:37.964497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:37.965873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:37.974020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 447/922] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:37.979377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.010826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936908f-11cd-4bce-9165-2f1f9cdd82e8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.043492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.044400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.044683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.044957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.045611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.046976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:38.059147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:38.060089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:38.060967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 476/923] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:38.061787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.084420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33812c-1d88-48b2-bb82-db8282963be1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.118050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.118850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.119009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.119256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.119957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:38.120281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.121672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.131445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 448/924] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 12 00:40:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:38.131445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.168228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683141e-f7ea-4cea-b063-65b01653da50 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.202092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.202894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.203173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.203443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.215588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.217002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:38.234398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:38.235351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:38.236858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:38.236858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:38.236858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:38.236858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:38.237099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c40eb-60b5-4dc7-a9a0-1d436427c340 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.237174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 477/925] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:38.237174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.275536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.276214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.276524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.276801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.277510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:38.277815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.279195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.288651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 449/926] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:38.288651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.335890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03d97-e4bb-417f-97a1-a39ff8f44e2e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.369276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.370036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.370218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.370468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.371201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.372602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.388603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157641-bbdc-4e71-85de-f9730c239600 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.391359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 478/927] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:38.397270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.436398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.437176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.437522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.437779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.438570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:38.438877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.440353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.458507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 450/928] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:38.460210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.525681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b51168-37c3-4705-b65e-93cd66cf3816 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.564384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.565128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.565410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.565748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.566477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:38.566788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.568307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.572119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c58527-ae3a-4690-84f1-fcd09d94e992 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.578862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 479/929] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:38.580394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.613267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.614093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.614372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.614648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.629118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.630826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:38.647200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:38.648206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:38.649202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 451/930] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:38.654505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.697201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdcc91-4dc8-4f38-a7fd-f35a3c6908b7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.733468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.734919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.735202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.735469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.736285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.737709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc90ba-ddfe-4c1e-bde7-7616f3601eef None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.763650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baa318-33cb-422e-8912-83c3ea8213d7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.764397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 480/931] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:38.766145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297bd57-6231-41c2-94df-f5d780de5550 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.766504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297bd57-6231-41c2-94df-f5d780de5550 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.766911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297bd57-6231-41c2-94df-f5d780de5550 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.768578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 481/932] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:38.774380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.774694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.774935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.793123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:38.794223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:38.814581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.815707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.816292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.816813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:38.817918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:38.818429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:38.821417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:38.842052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 452/933] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:38.847387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.872684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 482/934] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:38.889671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:38.956477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73895-73a7-45fb-9796-b0b8ac4bd3b6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.993267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc291a5-ec6d-43d6-b9b3-1c9da68c15ae None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:38.994255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:38.995064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:38.995346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:38.995626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.010482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.011969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:39.033053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:39.033993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:39.035107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:39.035107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:39.035107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:39.035107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:39.035107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 453/935] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:39.040020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.044558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.045318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.045598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.045865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.046594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.048048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:39.060761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:39.061785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:39.062623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 483/936] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:39.068163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.148411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea1133-f2dd-4bc0-858c-cdda8721df48 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.185695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.186807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.187343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.187835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.188827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:39.189336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.190935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.197101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b253f-2de7-42a0-8249-572f2b97f6a5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.204226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 454/937] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:39.210058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.235654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.236347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.236643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.236967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.237684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:39.237990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.239375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.245939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 484/938] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:39.251702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.328751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4865b-1c84-4cb8-abfd-dc901e7dd46a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.352931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d730-c9eb-4dca-928d-23c524ad1087 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.363612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.364792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.365289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.365752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.373012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.374710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.386865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.387597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.387882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.388157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.388900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:39.389206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.390632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:39.391523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:39.392379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:39.393387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:39.395706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 455/939] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:39.402526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.403113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 485/940] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:39.403835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.508355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f249e-0d97-453d-8810-ab40a3519be2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.510088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770004d-7c56-47a3-ba70-e92fa9b56e20 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.543376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.544019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.544338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.544672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.545478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] RBAC: Adding query filter params (interface=public, service_id=715bb3dc8c054c3ca480003927678889, region_id=RegionOne) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:39.545771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.547154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.547983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.549018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.549505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.550023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.551078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:39.551632 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.553306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.556297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 486/941] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/endpoints?service_id=715bb3dc8c054c3ca480003927678889&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:39.561138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.575728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 456/942] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:39.576676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:39.679929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9ef2b-5f76-48f4-b088-19757ed8e003 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.693145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4ec71-fca5-4317-97c9-3e77f763c381 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:39.720550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.722043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.722526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.723108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.724271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.726186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63317ef2-2273-4e96-9539-56712ed867d8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.728775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:39.729746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:39.730101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:39.730448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/7f17488343744b5cb36683c882d4bf21/roles/d41561fab1b144bb982fce5f82ef571d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:39.732750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=7f17488343744b5cb36683c882d4bf21, role_id=d41561fab1b144bb982fce5f82ef571d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:39.733802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 487/943] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:39 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:40:39.740794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366bfb9-a6a4-406e-9c28-1299e6c46cb0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:39.760075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 457/944] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:39 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/7f17488343744b5cb36683c882d4bf21/roles/d41561fab1b144bb982fce5f82ef571d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:40:40.714531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e3e5-1141-40ca-868b-61bc50b9d2d7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:40.715067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e3e5-1141-40ca-868b-61bc50b9d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:40.715067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e3e5-1141-40ca-868b-61bc50b9d2d7 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:40.716694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 488/945] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:40.722079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:40.722469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:40.722630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:40.739312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:40.740507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:40.808468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 458/946] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:40.831401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e3e5-1141-40ca-868b-61bc50b9d2d7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:40.943411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e3e5-1141-40ca-868b-61bc50b9d2d7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:40.977490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:40.978071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:40.978348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:40.978622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:40.979336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:40.980791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:40.982176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING keystone.server.flask.common [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 12 00:40:41.003460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.sql.core [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 00:40:41.003460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 12 00:40:41.003460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [parameters: {'id': 'c6b30119727f4a99afc63f359635d8f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 12 00:40:41.003460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=96826) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-77275400-83f9-49ac-a516-7600d9494b22 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6b30119727f4a99afc63f359635d8f8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6b30119727f4a99afc63f359635d8f8. May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application conn.query(q) May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:40:41.011682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result.read() May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 12 00:40:41.013477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return project_ref.to_dict() May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application next(self.gen) May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application yield resource May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application next(self.gen) May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.session.rollback() May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.force_reraise() May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise self.value May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application session.commit() May 12 00:40:41.015651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._prepare_impl() May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.session.flush() May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._flush(objects) May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application compat.raise_( May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application flush_context.execute() May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 00:40:41.017825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rec.execute(self) May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application c = connection._execute_20( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 00:40:41.020075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application conn.query(q) May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application result.read() May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:40:41.025086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [parameters: {'id': 'c6b30119727f4a99afc63f359635d8f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.025947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:41.026757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.Conflict( May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6b30119727f4a99afc63f359635d8f8. May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.027624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 489/947] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 12 00:40:40 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:41.028178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:41.102229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e1f4e-f63d-403b-934a-98738551cc72 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.102935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e1f4e-f63d-403b-934a-98738551cc72 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.103402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e1f4e-f63d-403b-934a-98738551cc72 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.108562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 490/948] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:41.113609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.116267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.116267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.126797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c0a8d-bb2a-4199-a503-93b180595d39 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:41.134605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:41.136746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:41.173097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:41.174590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.175071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.175726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.183523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:41.185157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:41.202445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:41.203517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:41.204654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:41.212736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 459/949] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:41.213745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:41.227798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 491/950] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:41.250222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:41.326700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b56cc2-af21-48ba-9cbe-52eeecfcd2b1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:41.363877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:41.364524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2c6e8-b777-44a8-9832-1054919ee328 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:41.364655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.364853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.365136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.366142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:41.366478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:41.368958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:41.391120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 460/951] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:41.406950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:41.407906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.408190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.408495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] PATH_INFO: `/v3/services/compute` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.409199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:41.410887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:41.424295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:41.425387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:41.426228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:41.430995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 492/952] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:41.432580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:41.543001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f0e-477b-416f-aa5d-ae959dd27478 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:41.579832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:41.580546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.580828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.581103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.581822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] RBAC: Adding query filter params (name=compute) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:41.582122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:41.583520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b88f49-14f9-4b5c-bccd-2f4e1f8952cf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:41.592330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 461/953] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:41.595022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:41.698822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090bf03-d239-45e3-af58-f58c5685ef54 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:41.731704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:41.733088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:41.733576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:41.734005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:41.734896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] RBAC: Adding query filter params (type=compute) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:41.735339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:41.736867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d61b2-185c-4de7-9df2-56ae01f3750d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:41.742110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 493/954] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:42.024748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b982c7-540a-4643-9759-644120fd1a8c None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.025845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b982c7-540a-4643-9759-644120fd1a8c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.026332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b982c7-540a-4643-9759-644120fd1a8c None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.028178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 462/955] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:42.033364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.033648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.033916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.050899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:42.052068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:42.117568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515467-e4e7-45b1-98d9-709c9aefb6b2 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.117921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515467-e4e7-45b1-98d9-709c9aefb6b2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.118102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515467-e4e7-45b1-98d9-709c9aefb6b2 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.119726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 463/956] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:42.125250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.125494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.125782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.126055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 494/957] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:42.146950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:42.148813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:42.153942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.215949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 464/958] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:42.238757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.251051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b8912-d329-4465-bff8-b082d1b35279 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.285725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.286726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.287006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.287438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.300485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:42.311248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:42.312217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:42.313093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 495/959] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:42.322226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.347404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b304041-dd4c-48f4-ab25-6cabbbf6c124 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.382085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.382984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.383270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.383540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.384276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.385652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:42.397858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:42.399917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:42.401025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 465/960] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:42.409057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.425756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97df20-3f15-4b13-aebb-7d61dacdc066 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.461863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.462869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.463323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.463775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.464723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:42.465224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.466775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.475621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 496/961] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:42.481606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.531656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d4ab-2794-42a6-8382-fbb17528b832 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.568691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.569649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.570104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.570544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.571413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:42.571856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.573394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.578567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 466/962] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:42.583778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.593234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab285-d6e8-475c-8ab8-16bae68ffbf4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.637029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.638580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.639375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.640115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.655316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.657824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:42.680544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:42.681502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:42.682395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:42.682395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:42.682395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:42.682395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:42.689331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 497/963] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:42.689973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.702509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd1c3-cddd-4af1-8b12-b2843a6a8257 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.736822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.737487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.737762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.738030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.738728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:42.739025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.740412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.750222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 467/964] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:42.751247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.812724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869eeca-f051-4c7f-8de2-07895e783620 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.853050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19512a18-693c-4487-b874-1cab8aeefdd6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:42.854217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.854928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.855362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.855656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.857200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:42.857526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.858906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.876286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 498/965] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:42.880044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:42.894296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:42.895134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:42.895480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:42.895819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:42.896761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:42.898424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:42.928545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 468/966] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:40:42.997674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619c59-b941-41a1-b2b4-62b2456a69bc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:43.040498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:43.041325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.041603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.041868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.049817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:43.051282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:43.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:43.068468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:43.069393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 499/967] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:40:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:43.075044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:43.187113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6876fc-b9c2-4ac8-a089-4e37ec84d42d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:43.220210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:43.220921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.221199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.221469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.222368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:43.222677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:43.224109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:43.242054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 469/968] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 12 00:40:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:43.247825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:43.347300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9d9a-05f6-4534-bca8-5df28043d796 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:43.382012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:43.383205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.383712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.384192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.453047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:43.454825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:43.552960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 500/969] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:43.560277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:43.672295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bc9-4ed1-49a1-88e5-a7e73e624c2e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:43.711566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:43.712319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.712605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.712880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.726276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:43.727762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6a01bf40-811d-4de6-a646-5c9f1b7707d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:43.764292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:43.765480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:43.767015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:43.767015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:43.767015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:43.767015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:43.767015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 470/970] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:43.769896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:43.874439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ef9b-13bf-4648-a649-95a9a26e1770 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.874778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ef9b-13bf-4648-a649-95a9a26e1770 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.875030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975ef9b-13bf-4648-a649-95a9a26e1770 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.883908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 471/971] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:43.885340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6a57-c08c-48d8-a812-35e91bbc1141 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.885627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6a57-c08c-48d8-a812-35e91bbc1141 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.885900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6a57-c08c-48d8-a812-35e91bbc1141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.899416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233caf-bb3e-49ad-96e6-c8ebd617c6c9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:43.913004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c4db6a57-c08c-48d8-a812-35e91bbc1141 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:43.914358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c4db6a57-c08c-48d8-a812-35e91bbc1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:43.935607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:43.936293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:43.936648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:43.936813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:43.937535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:43.937835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:43.939201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:43.951378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 501/972] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:43.956804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.016461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 472/973] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:44.027319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e862-d9c2-4b75-8c36-1fd3e476657c None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.027617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e862-d9c2-4b75-8c36-1fd3e476657c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.027897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e862-d9c2-4b75-8c36-1fd3e476657c None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.030868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 473/974] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:44.034585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.034876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.035158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.061403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:44.062657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:44.095173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0389-9694-44d8-86ab-1d60b46ffd17 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.139563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.140783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.141287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.141758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.143349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 474/975] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:44.144371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.166672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.168149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:44.183053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:44.184040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 502/976] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:44.185694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.266049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2c49d-d6ea-443a-bd0c-74820dad27e5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.293023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9730-8959-45fd-8388-7ce851beeb72 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.299230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.299912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.300197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.300499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.301201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.302649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.304071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING keystone.server.flask.common [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 12 00:40:44.318246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.sql.core [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 00:40:44.318246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 12 00:40:44.318246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [parameters: {'user_id': '0d9ba9ff388348ee895de5f030e0bbb6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 12 00:40:44.318246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=96825) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 00:40:44.326241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.326909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.327185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.327456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.328173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.329552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application conn.query(q) May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application result.read() May 12 00:40:44.334326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:40:44.335655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application next(self.gen) May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application yield resource May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application next(self.gen) May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.session.rollback() May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.force_reraise() May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise self.value May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application session.commit() May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 00:40:44.336913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._prepare_impl() May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.session.flush() May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._flush(objects) May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application compat.raise_( May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application flush_context.execute() May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rec.execute(self) May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 00:40:44.337807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application result = connection._execute_20( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 00:40:44.338697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application result = self._query(query) May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application conn.query(q) May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application result.read() May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 00:40:44.339536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [parameters: {'user_id': '0d9ba9ff388348ee895de5f030e0bbb6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:44.340861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 12 00:40:44.341782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 475/977] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 12 00:40:44 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:44.343125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.363285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 503/978] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:44 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:44.363285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.478891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40496-858e-4367-bb5d-55130c818fc1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.495265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb337b-73b4-4d71-a667-bb6d11a99b78 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.517535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.518499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.518740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.519108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.520021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:44.520452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.522466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.540145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 476/979] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:44.541019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.541840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.542117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.542385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.546849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.559852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.561627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:44.578991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:44.579989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:44.581282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 504/980] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:44.588807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:44.676812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c76f3c-e2fa-41e4-bab8-0f1f4562a13c None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.701587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123823c-2fb5-491c-8c17-e839341fbe70 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:44.712266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.713240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.713754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.714266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.715291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] RBAC: Adding query filter params (role.id=d41561fab1b144bb982fce5f82ef571d, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=7f17488343744b5cb36683c882d4bf21) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:44.715814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.717467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e53ac-0e83-4bcb-abf8-febaec4be6dc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.723844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 477/981] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:44 2022] GET /identity/v3/role_assignments?user.id=7f17488343744b5cb36683c882d4bf21&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=d41561fab1b144bb982fce5f82ef571d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:44.739181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:44.739885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:44.740174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:44.740487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:44.741323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:44.741738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:44.743744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e22a0-9ea1-4c4e-a95d-06d46d6b9af0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:44.787123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 505/982] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 12 00:40:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:45.339521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599bf9f-8975-4cf2-b91b-d395b5a570ae None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:45.340049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599bf9f-8975-4cf2-b91b-d395b5a570ae None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:45.340049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599bf9f-8975-4cf2-b91b-d395b5a570ae None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:45.341626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 478/983] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:45.347016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:45.347463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:45.347544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:45.364777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:45.365935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:45.432084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 506/984] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:45.453786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599bf9f-8975-4cf2-b91b-d395b5a570ae None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:45.558231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599bf9f-8975-4cf2-b91b-d395b5a570ae None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:45.591456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:45.592270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:45.592567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:45.592836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:45.593484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:45.594846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:45.606775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:45.607922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:45.608898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:45.612341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 479/985] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:45.614079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:45.707512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e992f6-0f80-4e49-9dbf-ef9b81a70a58 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:45.739268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:45.739929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:45.740202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:45.740499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:45.741198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:45.741565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:45.742933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:45.747810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 507/986] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:45.753686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:45.846471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42cb1-5f36-4301-8ffc-54685e8abaf3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:45.878603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:45.879265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:45.879542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:45.879812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:45.880544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:45.880848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:45.882317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:45.887512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 480/987] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:45.893204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:45.990863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a98c-1053-4035-9a64-9f118312a3fa None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:46.024119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:46.024806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:46.025090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:46.025360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:46.026181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] RBAC: Adding query filter params (interface=admin, service_id=715bb3dc8c054c3ca480003927678889, region_id=RegionOne) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:46.026484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:46.027835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a197c30-60ac-4037-80eb-5aa1dfbd9b19 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:46.033049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 508/988] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 12 00:40:45 2022] GET /identity/v3/endpoints?service_id=715bb3dc8c054c3ca480003927678889&interface=admin®ion_id=RegionOne => generated 193 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:46.038614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:46.151569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63906-bd71-4cbe-87d7-47ba589bf8cc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:46.185203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:46.185861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:46.186133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:46.186401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:46.187078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:46.188470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db5531-eb73-4fa8-91f2-d9c21ec58fb4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:46.194763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 481/989] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:46 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:40:46.643859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c1d52-5bc4-4d9b-81d0-ef9e77ada606 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:46.644414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c1d52-5bc4-4d9b-81d0-ef9e77ada606 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:46.644530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c1d52-5bc4-4d9b-81d0-ef9e77ada606 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:46.645855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 509/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:46.651265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271279-71b0-4887-92db-13aff914c427 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:46.651703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271279-71b0-4887-92db-13aff914c427 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:46.651780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271279-71b0-4887-92db-13aff914c427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:46.669814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9b271279-71b0-4887-92db-13aff914c427 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:46.671297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9b271279-71b0-4887-92db-13aff914c427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:46.739267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 482/991] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:46.762393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c1d52-5bc4-4d9b-81d0-ef9e77ada606 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:46.872497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c1d52-5bc4-4d9b-81d0-ef9e77ada606 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:46.908359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:46.909085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:46.909388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:46.909645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] PATH_INFO: `/v3/services/compute` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:46.910287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:46.911659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a5540154-ef02-49a1-8e52-72ac0dd9241c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:46.924125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:46.925284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:46.926170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 510/992] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:46.933293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271279-71b0-4887-92db-13aff914c427 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.012080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a9e8-e34e-4b56-b65b-bd518d724a8c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.012407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a9e8-e34e-4b56-b65b-bd518d724a8c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.012683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a9e8-e34e-4b56-b65b-bd518d724a8c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.014061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 511/993] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:47.019142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.019455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.019734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.036408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:47.037621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:47.040135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271279-71b0-4887-92db-13aff914c427 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.076221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.076923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.077208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.077484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.078203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:47.078510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.080131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.084746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 483/994] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:47.090551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.106674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 512/995] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:47.129143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.186055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3f96-ea4d-4080-a12d-d437cbac11f4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.218876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.219544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.219828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.220099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.220828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] RBAC: Adding query filter params (type=compute) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:47.221134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.222507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.225690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85ceb-f9cb-467f-adc0-b507f4e1190d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.227214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 484/996] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:47.232939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.263972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.264812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.265090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.265366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] PATH_INFO: `/v3/services/image` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.266008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.267367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:47.278937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:47.279956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:47.280851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 513/997] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:47.286126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.333317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd8c1-5ebb-4a49-9698-c16a7acc4b69 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.366427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.367224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.367511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.367792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.368653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] RBAC: Adding query filter params (interface=public, service_id=73789b504974426d9ac8ea82a083ecf9, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:47.368971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.370358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.375684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 485/998] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/endpoints?service_id=73789b504974426d9ac8ea82a083ecf9&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:47.381260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.384270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b581c7-3b13-451a-8193-0176d73b1fa5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.416684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.417354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.417661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.417933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.418638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] RBAC: Adding query filter params (name=image) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:47.418941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.420320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.425242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 514/999] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:47.431097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:47.478458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5483c6-d827-4e8c-b862-e3673ecf50d8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.511121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.512131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.512443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.512724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.513410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.514795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bb38d-5502-4b71-9c00-60cd16a2b19f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.521145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 486/1000] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/services => generated 1727 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:40:47.527717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdeaf18-0db5-40fb-a24b-db74da20ce58 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:47.565290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:47.565970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:47.566248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:47.566519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:47.567241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] RBAC: Adding query filter params (type=image) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:47.567531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:47.568969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dae558-34c2-4ecb-8a61-d56f1a838a37 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:47.584293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 515/1001] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:48.400227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2e2e2-505a-482e-b7eb-d729adb80318 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:48.400787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2e2e2-505a-482e-b7eb-d729adb80318 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:48.400787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2e2e2-505a-482e-b7eb-d729adb80318 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:48.402241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 487/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:48.407431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:48.407845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:48.407920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:48.425299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:48.426475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:48.492383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 516/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:48.514851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2e2e2-505a-482e-b7eb-d729adb80318 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:48.618515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2e2e2-505a-482e-b7eb-d729adb80318 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:48.653358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:48.654096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:48.654375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:48.654655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:48.655313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:48.656759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:48.669581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:48.670648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:48.676938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 488/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 12 00:40:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:48.677758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:48.780394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c352a-8180-43bf-9cbc-95c9f247ee50 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:48.812889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:48.813559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:48.813833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:48.814101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:48.814802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:48.815105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:48.816503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:48.821618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 517/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:48.827177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:48.926759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e9cc-f015-4c7f-9f06-bf13c447415d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:48.960148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:48.960831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:48.961102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:48.961370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:48.962064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:48.962361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:48.963725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513baa7b-57d1-49f0-94e6-a538f48d80f3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:48.968634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 489/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 12 00:40:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:48.974100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:49.071949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb85092-3323-4164-94e7-0050232fd6e1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:49.102553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:49.103212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:49.103493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:49.103764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:49.104538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:49.105907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13bc44-4dc2-49f9-9720-1169f902cdec None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:49.128292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 518/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:49.793835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0ef8-7771-4da5-a4a0-0d4f23142958 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:49.794393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0ef8-7771-4da5-a4a0-0d4f23142958 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:49.794393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0ef8-7771-4da5-a4a0-0d4f23142958 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:49.796192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 490/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:49.801801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:49.802280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:49.802466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:49.820972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:49.822162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:49.857775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dbc4f-c39a-4614-a729-a65fcfed7f4e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:49.858054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dbc4f-c39a-4614-a729-a65fcfed7f4e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:49.858324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dbc4f-c39a-4614-a729-a65fcfed7f4e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:49.859708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 491/1009] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:49.865078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae895-5035-447b-abca-484863cba789 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:49.865345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae895-5035-447b-abca-484863cba789 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:49.865607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae895-5035-447b-abca-484863cba789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:49.882644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-11bae895-5035-447b-abca-484863cba789 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:49.883850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-11bae895-5035-447b-abca-484863cba789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:49.899939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 519/1010] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:49.922201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:49.960872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 492/1011] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:49.984030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bae895-5035-447b-abca-484863cba789 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:50.021418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369dd6-df76-47c8-a943-8118a4a44760 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:50.055269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:50.055969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:50.056283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:50.056554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] PATH_INFO: `/v3/services/compute` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:50.057199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:50.058576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:50.070823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:50.071829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:50.075149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 520/1012] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:50.078252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:50.089921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bae895-5035-447b-abca-484863cba789 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:50.125264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:50.125932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:50.126213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:50.126491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:50.127569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:50.128968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:50.148209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 493/1013] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:49 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:50.193657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7b2a-63f4-415a-8c9a-a59bbdb24feb None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:50.236182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:50.236911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:50.237184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:50.237453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:50.238169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] RBAC: Adding query filter params (name=compute) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:50.238472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:50.239894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:50.245740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 521/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:50.252507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:50.357725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393a290-cd54-45f0-a794-7f68441ef311 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:50.393361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:50.394046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:50.394329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:50.394610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:50.395333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] RBAC: Adding query filter params (type=compute) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:50.395638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:50.397054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5790b2-ce6f-4654-a908-9bcaad91d92f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:50.407058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 494/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 12 00:40:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:50.408456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:50.507267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697310-4aea-4867-b5d6-96de8f3de501 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:50.541466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:50.542135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:50.542415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:50.542681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:50.543426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:50.544896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b02b2-481f-4961-b0d7-aa87f4c13535 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:50.572358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 522/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:51.378185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da67-7c87-46af-97fd-8b24f1a2503b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:51.378723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da67-7c87-46af-97fd-8b24f1a2503b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:51.378723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da67-7c87-46af-97fd-8b24f1a2503b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:51.380273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 495/1017] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:51.385643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:51.386077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:51.386153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:51.403663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:51.404758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:51.478269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 523/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:51.501362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0da67-7c87-46af-97fd-8b24f1a2503b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:51.601934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0da67-7c87-46af-97fd-8b24f1a2503b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:51.635104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:51.635896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:51.636174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:51.636475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:51.637644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:51.639074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a42b5-24c9-4f90-92e3-8258795250d3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:51.640870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 496/1019] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:51.646555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:51.745250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dca3fb-0032-4a7e-aa1c-65213955200f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:51.781914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:51.782562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:51.782846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:51.783160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:51.783888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:51.785389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16d4d3-60fd-4fa3-aa28-45631a07c4c4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:51.812645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 524/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:40:51 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:40:52.387411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276d9e-c275-4ee9-a434-501e4db16356 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.387931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276d9e-c275-4ee9-a434-501e4db16356 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.387931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37276d9e-c275-4ee9-a434-501e4db16356 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.389516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 497/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:52.394916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.395369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.395448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.414155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:52.415229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:52.484041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 525/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:52.506707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37276d9e-c275-4ee9-a434-501e4db16356 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:52.611638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37276d9e-c275-4ee9-a434-501e4db16356 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:52.646577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:52.647465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.647743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.648103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] PATH_INFO: `/v3/services/image` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.648793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:52.650175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:52.661936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:52.662938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:52.664352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 498/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 12 00:40:52 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:52.671362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:52.776466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f874461-f539-4b32-8c08-aeea1874c6cd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:52.810906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:52.811567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.811891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.812170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.812883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] RBAC: Adding query filter params (name=image) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:52.813185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:52.814544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74183051-3214-4ad3-a019-4232f2b8910e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:52.819777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 526/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:52.825521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:52.909200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf645-91b5-476f-9498-39a6c3bc6299 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.909551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf645-91b5-476f-9498-39a6c3bc6299 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.909738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbf645-91b5-476f-9498-39a6c3bc6299 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.911294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 527/1025] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:52.917235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.917459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.917741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.925921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344c3cc-c04b-4f73-8d8a-67926f95cdf4 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:52.934499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:52.935797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:52.961162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:52.961853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:52.962129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:52.962402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:52.963130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] RBAC: Adding query filter params (type=image) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:52.963432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:52.964877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:52.974967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 499/1026] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:52.976754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.008305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 528/1027] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:53.026536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.080901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d97d-a67f-4cea-bc86-1cff392d3492 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.115259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.115931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.116590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.116874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.117663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] RBAC: Adding query filter params (interface=public, service_id=b2f857686b2a4ff5a5a79635a403c71c, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:53.117973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.119352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.124391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 500/1028] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:40:52 2022] GET /identity/v3/endpoints?service_id=b2f857686b2a4ff5a5a79635a403c71c&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:53.130220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.132286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392bf1-932a-4f7a-ab20-90604ab3a039 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.175416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.176490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.176770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.177304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.190249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:53.201378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:53.202435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:53.208470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 529/1029] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:53.211543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.232211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a22b79-3117-431d-8907-f088c2efa0d1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.266777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.267440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.267725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.267998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.268716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.270106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.276503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 501/1030] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:40:53.319418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d094ca-be77-47c4-804b-d14b4fd4aa80 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.360419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.361376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.361800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.362263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.363257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:53.363705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.365266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.374873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 530/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:53.380972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d01fd-fc12-4c48-8df7-4fa66216b471 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.520798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.522081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.522413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.522699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.533563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.535025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:53.551960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:53.552935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:53.553909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:53.553909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:53.553909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:53.553909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:53.553909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 502/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:53.559352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.658877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8944ba-fa36-475b-bcb8-242b973cb8be None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.691339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.692000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.692283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.692574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.693291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:53.693590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.695080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.707474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 531/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:53.713238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.813648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08d77f-2fbd-4dbc-80c4-9250283c2b40 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:53.846964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:53.847757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:53.848033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:53.848300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] PATH_INFO: `/v3/users/nova` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:53.855953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:53.857417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:53.871141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:53.872160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:53.873614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:53.876700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 503/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:53.878713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:53.981777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251133f-e17e-4e6c-a8d0-f352e0a9d2ba None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.015846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.016554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.016830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.017100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.017922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:54.018227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.019638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92031915-d650-4390-a53c-312cd8a68723 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.041676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 532/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:54.042981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.070408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2beb9b-d211-4679-98d2-fed1dc971eda None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.070408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2beb9b-d211-4679-98d2-fed1dc971eda None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.070679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2beb9b-d211-4679-98d2-fed1dc971eda None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.072351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 533/1036] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:54.077354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.077655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.077920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.094062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:54.095228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:54.154782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92654ae-deea-4581-84cc-73d0c154d5be None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.161260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 534/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:54.185314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.186416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.187063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.187343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.187612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.254578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.256029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.279761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43158ba6-edb3-4bd9-b1c5-29c1e3c08009 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.312670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.313521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.313808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.314082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.315263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.316725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.318499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 535/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:54.324103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.352076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 504/1039] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:54.357297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.419320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40c21-62b9-4799-a365-1acdd848a336 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.452041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda514-10a3-444b-972f-c1466076e994 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.452846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.453806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.454322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.454653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.455433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.456950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.487818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 536/1040] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:40:54 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:40:54.491706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.492733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.493017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.493291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.505480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.507463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:54.526488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:54.527457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:54.528369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:54.528369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:54.528369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:54.528369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:54.528369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 505/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:54.533586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.640876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061014-d57b-4788-baff-a6666b1ac99c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.675355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.676040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.676344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.678126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.678126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:54.678126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.679183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.692215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 537/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:54.700000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.801215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96529e35-74d3-4f7f-8819-304aef7b4abf None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:54.837065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:54.837879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:54.838158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:54.838428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:54.845181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:54.846627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:54.860645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:54.861539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:54.862446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 506/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:54.868746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:54.969158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22a7a1-5472-4aca-b63c-ee0ff9d50cc8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:55.003611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:55.004276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:55.004583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:55.004855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:55.005643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:55.005941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:55.007684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989263e8-d5ad-472c-a7b1-d99d29b03282 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:55.019261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 538/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:55.025918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:55.130094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dca85-a1ae-47ca-9197-580728408cc2 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:55.168973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:55.169635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:55.169912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:55.170178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:55.171011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] RBAC: Adding query filter params (role.id=d41561fab1b144bb982fce5f82ef571d, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:55.171317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:55.172797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fa666-074e-48cb-88f3-931ef35db6d8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:55.184334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 507/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:55 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=d41561fab1b144bb982fce5f82ef571d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:55.547628 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f17b6-67f7-4a26-9a71-8d33a84eaa1c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:55.547872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f17b6-67f7-4a26-9a71-8d33a84eaa1c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:55.548173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f17b6-67f7-4a26-9a71-8d33a84eaa1c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:55.549747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 539/1046] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:55.555094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:55.555554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:55.555803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:55.573625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:55.574811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:55.652668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 508/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:55.673137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f17b6-67f7-4a26-9a71-8d33a84eaa1c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:55.808454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f17b6-67f7-4a26-9a71-8d33a84eaa1c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:55.846589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:55.847485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:55.847793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:55.848066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] PATH_INFO: `/v3/services/image` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:55.848750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:55.850144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:55.865826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:55.867096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 00:40:55.867943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:55.872643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 540/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 12 00:40:55 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:55.874339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:55.981693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8f88a-4d9c-42a9-80f8-55023aaf5d17 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:56.019659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:56.020364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:56.020650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:56.020925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:56.021635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] RBAC: Adding query filter params (name=image) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:56.021942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:56.023934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:56.029439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 509/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:56.035174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:56.152526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af021-9976-4315-9f4c-a260801706e0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:56.186999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:56.187676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:56.187954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:56.188225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:56.188958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] RBAC: Adding query filter params (type=image) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:56.189261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:56.190635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93770a5-5ff0-454a-be23-fd44dbabcf62 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:56.202333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 541/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:56.204060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:56.318859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192b2f6-e2bd-4aca-8898-9d8a34cb8f49 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:56.359089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:56.360533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:56.360533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:56.360533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:56.361069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:56.362442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6dfd8-9efa-4bc4-b5a5-c8762bf1a038 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:56.384545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 510/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:40:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:40:56.914885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5f48-6709-44ba-93e5-40ece4c39d8e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:56.915477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5f48-6709-44ba-93e5-40ece4c39d8e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:56.915477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5f48-6709-44ba-93e5-40ece4c39d8e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:56.917047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 542/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:56.922419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:56.922872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:56.922950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:56.940107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:56.941310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:57.015799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 511/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:40:57.038876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff5f48-6709-44ba-93e5-40ece4c39d8e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.150896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff5f48-6709-44ba-93e5-40ece4c39d8e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.184736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.185883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.186369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.186835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.197730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:57.205624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:57.206565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:40:57.207408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:57.214973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 543/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:57.216276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.317245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98949-94f7-4a37-85dc-6a8c8d681d19 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.353867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.354543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.354820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.355093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.355854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:57.356148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:57.357579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b279-7287-4369-8954-5a351d804a8b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:57.366251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 512/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:57.372044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.467061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bedf3-bb7e-4f37-8c19-f52795548577 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.488988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7d9-08b4-46ac-aa7c-065b95f3ae52 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.489287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7d9-08b4-46ac-aa7c-065b95f3ae52 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.489542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88f7d9-08b4-46ac-aa7c-065b95f3ae52 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.491450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 513/1056] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:40:57.496135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.496453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.496728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.499993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.500909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.501193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.501449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] PATH_INFO: `/v3/users/4a0679d510bf4f64af508f1648c21bfc` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.514656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:57.515837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:57.516402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] RBAC: Authorizing `identity:get_user(user_id=4a0679d510bf4f64af508f1648c21bfc)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:57.517848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:57.519707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 544/1057] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/users/4a0679d510bf4f64af508f1648c21bfc => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:57.525365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.585546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 514/1058] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:57.607889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.623525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710e0e-8c91-4fe6-9736-c38d8683281d None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.657250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.657893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.658172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.658441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.704376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67267e-fe5d-4c3d-8b2b-272c93285f07 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.722822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:57.724235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:57.736898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.737726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.737994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.738261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.748672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:40:57.757230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:57.758138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:40:57.759013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:57.760573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 515/1059] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:57.764434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.832465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 545/1060] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 12 00:40:57.833441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.861711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1de4e-915e-4205-8525-7be054cb880d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.898515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.899568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.900001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.900451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.901628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:57.902108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:57.904304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:57.916553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 516/1061] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:57.922743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:57.939692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf64a30-d68d-45f0-82ef-d36727a1f1a4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:57.975063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:57.976210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:57.976210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:57.976447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:57.977634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:57.979076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:57.981222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 546/1062] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:57.986840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.031297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a51d47-235e-4e8b-9eda-eb9c9414023b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.063337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.064134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.064460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.064713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.076515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.077966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.085211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897e03f-ec3c-40ff-ae59-a65315c26623 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:58.094721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:58.095660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:58.096910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:58.096910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:58.096910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:58.096910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:58.096910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 517/1063] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:58.101804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.122652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.123316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.123591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.123869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.124726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7, user.id=4a0679d510bf4f64af508f1648c21bfc) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:58.125050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.126483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.134438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 547/1064] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:40:57 2022] GET /identity/v3/role_assignments?user.id=4a0679d510bf4f64af508f1648c21bfc&scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:58.218118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a2dfb-7dcb-425c-be8c-0e6614e62844 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.255527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.256109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.257058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.257058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.257667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:58.258058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.259467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.276348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 518/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:58.279099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.381963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d70cc-27d0-4992-ae0c-d2192e4dcaec None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.415299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.416092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.416399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.416669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.423646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.425099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:40:58.439936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:58.440928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:58.441873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 548/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:58.448293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.551092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18faa27-cccc-4098-b0a6-85093ccb00e6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.585215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.586093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.586366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.586640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.587455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:58.587755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.589217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def58205-2e9e-4645-88bc-d1e07cd63e3b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.611003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 519/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:58.612767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.650864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651e85-3a11-4eed-ad4f-05dae1e69c09 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.651196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651e85-3a11-4eed-ad4f-05dae1e69c09 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.651424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651e85-3a11-4eed-ad4f-05dae1e69c09 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.652848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 520/1068] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:40:58.657994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.658367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.658653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.674876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:40:58.676054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:40:58.710935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be513c8-11c7-4983-966f-972e0356bd10 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.741830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 521/1069] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:40:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:40:58.744384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.745182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.745459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.745700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.756338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.757790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.763363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:58.775402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:58.776350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:58.777265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:58.777265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:58.777265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:58.777265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:58.777265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 549/1070] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:40:58.782229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.863810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bbb2b-ad4d-4548-81db-404d06f316ef None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.879054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89c801-16fb-41c3-bd07-e43a8dd97c6f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:58.899368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.900025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.900337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.900616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.914725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:58.915467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:58.915739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:58.916007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:58.916814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:58.917097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.918490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:58.935193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 550/1071] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:58.936406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:58.967002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:58.968480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.034553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30bbe8-e9ed-4290-b9b6-05c55bcde8a4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:59.063360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 522/1072] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:40:59.069499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:59.073806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:59.074624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:59.074899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:59.075166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:59.083851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:59.085693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:40:59.099622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:59.100803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 551/1073] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:59.106824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:59.166352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a72f-a940-49e2-adaa-94f67dc90ce9 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:59.199828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:59.200671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:59.200960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:59.201229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:59.204239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb34fd-2afb-407f-8779-47503c293d5f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:59.211196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:59.212685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:40:59.231006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:40:59.231990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:40:59.233064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:40:59.233064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:40:59.233064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:40:59.233064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:40:59.233064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 523/1074] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:40:59.238138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:59.245586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:59.246538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:59.246962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:59.247375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:59.248375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:59.248830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:59.250488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.268580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 552/1075] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:40:59.269640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:40:59.334959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435071de-228c-4946-8d15-2fb67ab6b8f1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:59.367859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4b8f-deeb-4e18-8087-639cb5fd2cc8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:40:59.369078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:59.369982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:59.370415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:59.370684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:59.371435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:40:59.371728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:59.373126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4ce1e-0103-4cf8-bcd0-08bba928c842 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.384780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 524/1076] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:40:59.402151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:40:59.402984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:40:59.403264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:40:59.403535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/d41561fab1b144bb982fce5f82ef571d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:40:59.405912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=736e16d1bc05411faea9c943f930acb1, role_id=d41561fab1b144bb982fce5f82ef571d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:40:59.407389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf0a5f-040b-49f7-8c79-76f861064cbe None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:40:59.423600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 553/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:40:59 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/736e16d1bc05411faea9c943f930acb1/roles/d41561fab1b144bb982fce5f82ef571d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:41:00.360897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea02a-9666-418e-94d9-65148007b73a None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:00.361452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea02a-9666-418e-94d9-65148007b73a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:00.361452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea02a-9666-418e-94d9-65148007b73a None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:00.363183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 525/1078] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:00.368516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491309-b045-4867-a596-89e84e658b62 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:00.368963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491309-b045-4867-a596-89e84e658b62 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:00.369073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491309-b045-4867-a596-89e84e658b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:00.386951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-04491309-b045-4867-a596-89e84e658b62 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:00.388142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-04491309-b045-4867-a596-89e84e658b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:00.454836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 554/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:00.478257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ea02a-9666-418e-94d9-65148007b73a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:00.592927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ea02a-9666-418e-94d9-65148007b73a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:00.626895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:00.627695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:00.628210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:00.628210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:00.638791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:00.646224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:00.647365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:00.648504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 526/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:00.653898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491309-b045-4867-a596-89e84e658b62 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:00.752391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491309-b045-4867-a596-89e84e658b62 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:00.785953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:00.786611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:00.786889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:00.787157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:00.787916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:00.788237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:00.789623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:00.797610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 555/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:00.803755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:00.905107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79d431-0a9b-42b4-9a38-77e1daec6b00 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:00.939628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:00.940564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:00.940846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:00.941124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] PATH_INFO: `/v3/users/4a0679d510bf4f64af508f1648c21bfc` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:00.942355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0679d510bf4f64af508f1648c21bfc)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:00.943798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:00.945515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 527/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:00 2022] GET /identity/v3/users/4a0679d510bf4f64af508f1648c21bfc => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:00.951442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:01.052531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d51e69-4677-4017-b40f-7ac214f50864 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:01.085647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:01.086516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:01.086792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:01.087060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:01.088220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:01.089682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5357ad-94f2-4313-bd33-dcc30d7fad01 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:01.091377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 556/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:00 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:01.097398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:01.199927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540a630-2ae0-40a2-ab5f-ee6847a777d1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:01.234216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:01.235104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:01.236805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:01.236805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7/users/4a0679d510bf4f64af508f1648c21bfc/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:01.237753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7, user_id=4a0679d510bf4f64af508f1648c21bfc, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:01.239255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777745f-90a7-408c-9452-d44f146bc689 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:01.253504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 528/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:41:01 2022] PUT /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7/users/4a0679d510bf4f64af508f1648c21bfc/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:41:01.677717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da939f-f660-489c-a040-ae396f839c21 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:01.678322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da939f-f660-489c-a040-ae396f839c21 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:01.678322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da939f-f660-489c-a040-ae396f839c21 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:01.679729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 557/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:01.685089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:01.685551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:01.685660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:01.703473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:01.704683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:01.773297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 529/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:01.796692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da939f-f660-489c-a040-ae396f839c21 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:01.905031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da939f-f660-489c-a040-ae396f839c21 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:01.940096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:01.940860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:01.941137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:01.941406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:01.952185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:01.960273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:01.961221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:01.962138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 558/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 12 00:41:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:01.967807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.076360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3276f9-038e-4caa-9564-947bf4b481b2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.110881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.111538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.111811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.112080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.112854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:02.113153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.114520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:02.122655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 530/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:41:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:02.128789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.227668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87654f-8a4f-4b25-bc09-471935497a4e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.263761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.264592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.264871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.265142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.274984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.276460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:41:02.294417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:02.295322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:02.296197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:41:02.296197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:41:02.296197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:41:02.296197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:02.296197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 559/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:02.301680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.405814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6de5-d673-4fda-bb11-724cbbdc6c08 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.440228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.440940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.441220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.441478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.442203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:02.442502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.443882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:02.456431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 531/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:02.462443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.565653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf530bf-4aec-4a36-88fa-792723b94773 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.601345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.602180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.602366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.602703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.609911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.611338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:41:02.625681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:02.627014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:02.628350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 560/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 12 00:41:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:02.633383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.729763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b797bc-219a-4dd2-9f92-12074db4c46f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.763343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.764000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.764278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.764572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.765404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:02.765707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.767123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:02.784110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 532/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 12 00:41:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:02.789955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:02.887384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3e266-e145-405d-9fc3-dd311b0122d9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:02.920995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:02.921636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:02.921915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:02.922183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:02.988395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:02.989832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.088603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 561/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:03.094701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.194652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e08365-4358-44f1-aa27-45cc581035dd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.229214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.230016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.230290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.230558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.241368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.243034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:41:03.261353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:03.262456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:03.263379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:41:03.263379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:41:03.263379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:41:03.263379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:03.263379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 533/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:03.268995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.369911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e8c8-8de2-49c8-975f-df44c759e59e None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.406528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.407141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.407421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.407691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.408438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:03.408741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.410108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440887b-3b68-400c-9afc-bd5ff17fb717 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.421205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 562/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:03.426945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.458491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2671-e5f6-460f-94b6-b70f511792ec None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.458752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2671-e5f6-460f-94b6-b70f511792ec None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.459035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2671-e5f6-460f-94b6-b70f511792ec None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.460504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 563/1096] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:41:03.465787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.466076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.466357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.483223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:03.484411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:03.527043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55880ec-0cac-452b-b684-17c43721f2ad None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.551538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 564/1097] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:03.562923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.563724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.564004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.564239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.571101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.572574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.574685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:41:03.589489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:03.590535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:03.591530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 534/1098] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:03.596728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.678051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc37fb-181c-4d69-9fcc-24fa33c0a797 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.693967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec712-fde9-4e26-8a9b-7b23324d6c1b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.714499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.715292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.715566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.715832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.725743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.725911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:03.726254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.726527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.726790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.727570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:03.727873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.729329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:03.733181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:03.734287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:03.735166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 565/1099] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:41:03.740567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.741125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 535/1100] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:03.746734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.833043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28474a-73df-4e53-986d-866e49e28642 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.846691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb52c-2901-459c-b38e-a4a4a75c7c3d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:03.865176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.865836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.866159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.866437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.867202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:03.867498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.868914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.877291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 566/1101] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:03.880826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:03.881481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:03.882099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:03.882218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:03.883041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:03.883407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] RBAC: Adding query filter params (role.id=d41561fab1b144bb982fce5f82ef571d, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=736e16d1bc05411faea9c943f930acb1) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:03.883842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:03.885622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:03.892427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 536/1102] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/role_assignments?user.id=736e16d1bc05411faea9c943f930acb1&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=d41561fab1b144bb982fce5f82ef571d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:03.984199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7b6a-519c-4792-8dfd-78cd589a5bac None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:04.023184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:04.024171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:04.024484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:04.024754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] PATH_INFO: `/v3/users/4a0679d510bf4f64af508f1648c21bfc` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:04.026015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0679d510bf4f64af508f1648c21bfc)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:04.027462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:04.029173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 567/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:03 2022] GET /identity/v3/users/4a0679d510bf4f64af508f1648c21bfc => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:04.035180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:04.138598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d339-35fd-428b-b47b-2c8382ae0012 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:04.175609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:04.176432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:04.176717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:04.176998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:04.245030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:04.246408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:04.342142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 537/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:04.347775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:04.446427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620a5c-40a0-43ac-b41f-e583091f8ad5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:04.480267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:04.481150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:04.481436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:04.481707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:04.482865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:04.484354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e3c04-f127-4eeb-9c75-44ece21f22bd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:04.486038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 568/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:04 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:04.492726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:04.593996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd73d-426e-4105-9e9d-133493632b32 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:04.627535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:04.628187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:04.628489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:04.628796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:04.629594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7, user.id=4a0679d510bf4f64af508f1648c21bfc) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:04.629901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:04.631320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10308ce-bbdc-437b-8cc4-01f505d2074d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:04.637357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 538/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:04 2022] GET /identity/v3/role_assignments?user.id=4a0679d510bf4f64af508f1648c21bfc&scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:06.842922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78485-b5ed-4ebf-bed2-65f67873debc None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:06.843797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78485-b5ed-4ebf-bed2-65f67873debc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:06.843797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78485-b5ed-4ebf-bed2-65f67873debc None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:06.846023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 569/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:06.853919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:06.854386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:06.854484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:06.872277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:06.873436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:06.939257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 539/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:41:06.973818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78485-b5ed-4ebf-bed2-65f67873debc None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:07.088762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78485-b5ed-4ebf-bed2-65f67873debc None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:07.127221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:07.128008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:07.128276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:07.128581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:07.129871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:07.131347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a6538-4cb2-4f33-89cd-5bcf9d232974 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:07.133402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 570/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:07.139496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:07.252928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b65441-4ec5-4b58-905e-c940c32f486f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:07.288290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:07.288974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:07.289257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:07.289530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:07.290238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:07.291674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161b674-01bc-48e5-a74f-80fec2805a39 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:07.316971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 540/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:41:07 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:09.565024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6e38-5ec9-4e57-8731-8f8f23e6f8e7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:09.565840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6e38-5ec9-4e57-8731-8f8f23e6f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:09.565840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae6e38-5ec9-4e57-8731-8f8f23e6f8e7 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:09.567946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 571/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:09.576547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:09.577086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:09.577086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:09.593990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:09.595144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:09.660760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 541/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:09.698052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae6e38-5ec9-4e57-8731-8f8f23e6f8e7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:09.809027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae6e38-5ec9-4e57-8731-8f8f23e6f8e7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:09.844108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:09.844906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:09.845190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:09.845447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] PATH_INFO: `/v3/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:09.847023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] RBAC: Authorizing `identity:get_role(role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:09.848424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:09.849776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 572/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:09 2022] GET /identity/v3/roles/041b2fa99a644a7aba7f82618674fc9d => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:09.859562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:09.962376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2631c-975a-47f3-b2bb-039d6d7cbf75 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:09.996047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:09.996932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:09.997210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:09.997480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] PATH_INFO: `/v3/users/2a4e923f596d4254804475299422a529` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:10.011277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] RBAC: Authorizing `identity:get_user(user_id=2a4e923f596d4254804475299422a529)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:10.012740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:10.014393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 542/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:09 2022] GET /identity/v3/users/2a4e923f596d4254804475299422a529 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:10.020460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:10.125976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8d79-a5ef-46b4-a272-162de8f05f3b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:10.160122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:10.160821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:10.161103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:10.161372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:10.225327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:10.226765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:10.328866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 573/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:10.336692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:10.438427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e363e5-012a-432f-9429-bb449a8f8d8e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:10.471863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:10.472745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:10.473020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:10.473287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:10.474570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:10.476171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1fdfd-9680-4612-b312-851d7fea6e14 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:10.477888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 543/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:10 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:10.484932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:10.599501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3430c3-a5d4-4fac-ac4c-2f4c25128226 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:10.635817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:10.636501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:10.636767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:10.637037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:10.637840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7, user.id=2a4e923f596d4254804475299422a529) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:10.638142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:10.639588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0e8a1-d532-4393-b8fd-de85ef820947 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:10.645661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 574/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:10 2022] GET /identity/v3/role_assignments?user.id=2a4e923f596d4254804475299422a529&scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:12.817643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6352b8-9806-4653-a825-dca6f5874e31 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:12.818203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6352b8-9806-4653-a825-dca6f5874e31 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:12.818203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6352b8-9806-4653-a825-dca6f5874e31 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:12.819510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 544/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:12.825979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:12.826673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:12.826866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:12.853757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:12.854952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:12.923533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 575/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:12.946153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6352b8-9806-4653-a825-dca6f5874e31 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:13.049192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6352b8-9806-4653-a825-dca6f5874e31 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:13.083754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:13.084560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:13.084836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:13.085162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] PATH_INFO: `/v3/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:13.086826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] RBAC: Authorizing `identity:get_role(role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:13.088200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:13.089590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 545/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:12 2022] GET /identity/v3/roles/041b2fa99a644a7aba7f82618674fc9d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:13.096076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:13.206822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c3bd1-9b65-4a8b-a881-2ecce7d0bea4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:13.247301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:13.248670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:13.249086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:13.249514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] PATH_INFO: `/v3/users/2a4e923f596d4254804475299422a529` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:13.251964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] RBAC: Authorizing `identity:get_user(user_id=2a4e923f596d4254804475299422a529)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:13.254326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:13.256787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 576/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:13 2022] GET /identity/v3/users/2a4e923f596d4254804475299422a529 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:13.262990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:13.363768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e996-e671-42a6-aa4d-c136ef7ab781 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:13.400488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:13.401341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:13.401616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:13.401885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:13.403179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:13.404656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11841065-2f90-4827-ada2-c7c67fa9f830 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:13.406351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 546/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:13 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:13.415283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:13.531421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f39bc-e97c-4640-b8d3-9f9788a7b4fd None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:13.567837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:13.568731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:13.569012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:13.569305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7/users/2a4e923f596d4254804475299422a529/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:13.571797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7, user_id=2a4e923f596d4254804475299422a529, role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:13.573264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a5e32-44e9-4a08-88e0-2859989bbcc8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:13.588497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 577/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:41:13 2022] PUT /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7/users/2a4e923f596d4254804475299422a529/roles/041b2fa99a644a7aba7f82618674fc9d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:41:15.749874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e5b7-fd67-48dc-ab01-3d396912ad6a None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:15.750730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e5b7-fd67-48dc-ab01-3d396912ad6a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:15.750730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4e5b7-fd67-48dc-ab01-3d396912ad6a None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:15.753409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 547/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:15.759299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:15.759736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:15.759838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:15.777045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:15.778198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:15.846557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 578/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:15.873530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e5b7-fd67-48dc-ab01-3d396912ad6a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:15.986467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4e5b7-fd67-48dc-ab01-3d396912ad6a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:16.021605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:16.022413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:16.022720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:16.022988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] PATH_INFO: `/v3/roles/041b2fa99a644a7aba7f82618674fc9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:16.024756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] RBAC: Authorizing `identity:get_role(role_id=041b2fa99a644a7aba7f82618674fc9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:16.026146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:16.027589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 548/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:15 2022] GET /identity/v3/roles/041b2fa99a644a7aba7f82618674fc9d => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:16.035202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:16.139782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a219f6-c392-4610-ad5f-e8f00a103cdf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:16.176280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:16.177161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:16.177439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:16.177707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] PATH_INFO: `/v3/users/2a4e923f596d4254804475299422a529` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:16.179090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=2a4e923f596d4254804475299422a529)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:16.180529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:16.182160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 579/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:16 2022] GET /identity/v3/users/2a4e923f596d4254804475299422a529 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:16.188155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:16.290900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b2997-a9fd-4f92-b44c-c29645d87b7d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:16.330005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:16.330654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:16.330931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:16.331200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:16.397649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:16.399094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:16.493579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 549/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:16.499800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:16.611334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6070d-f328-4761-aab8-cd8f922b7ce5 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:16.649020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:16.649876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:16.650154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:16.650426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] PATH_INFO: `/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:16.651618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] RBAC: Authorizing `identity:get_project(project_id=1b4bf127de1c41c491e0b1f7f66c4bb7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:16.653100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4371b-8ce7-4542-b866-9363ec8ad39f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:16.654798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 580/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:16 2022] GET /identity/v3/projects/1b4bf127de1c41c491e0b1f7f66c4bb7 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:16.661506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:16.761144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c7bb-c607-476d-8f36-3fe7e10996e3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:16.795718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:16.796378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:16.796661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:16.796933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:16.797762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] RBAC: Adding query filter params (role.id=041b2fa99a644a7aba7f82618674fc9d, scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7, user.id=2a4e923f596d4254804475299422a529) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:16.798073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:16.799499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1725f1-2c72-49b1-a6cf-e4bbdf2240b7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:16.805455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 550/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:16 2022] GET /identity/v3/role_assignments?user.id=2a4e923f596d4254804475299422a529&scope.project.id=1b4bf127de1c41c491e0b1f7f66c4bb7&role.id=041b2fa99a644a7aba7f82618674fc9d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:19.006740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9327f7-6311-43bb-bef7-d3d9c2a28cdf None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:19.007308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9327f7-6311-43bb-bef7-d3d9c2a28cdf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:19.007308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9327f7-6311-43bb-bef7-d3d9c2a28cdf None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:19.008682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 581/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:19.014932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:19.015369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:19.015496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:19.036996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:19.038223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:19.112956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 551/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:19.137180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9327f7-6311-43bb-bef7-d3d9c2a28cdf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:19.258597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9327f7-6311-43bb-bef7-d3d9c2a28cdf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:19.291717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:19.292525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:19.292807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:19.293074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:19.294288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:19.295829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5f5c3-22d4-4805-9fa6-53e176435dcb None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:19.297474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 582/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:19.304142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:19.409023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d6f9-8b18-4854-acee-c09eaed7c84e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:19.442873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:19.443522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:19.443803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:19.444072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:19.444997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:19.446425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6ef69-df9b-4185-939b-d33c90f6a81b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:19.477893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 552/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:41:19 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:21.692070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706331-8acb-4d5d-a73e-a905c7c3c98c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:21.692978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706331-8acb-4d5d-a73e-a905c7c3c98c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:21.692978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706331-8acb-4d5d-a73e-a905c7c3c98c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:21.694947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 583/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:21.700718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:21.701152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:21.701264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:21.718459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:21.719615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:21.786118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 553/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:41:21.811460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86706331-8acb-4d5d-a73e-a905c7c3c98c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:21.915641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86706331-8acb-4d5d-a73e-a905c7c3c98c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:21.949663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:21.950377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:21.950663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:21.950940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:21.952232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:21.953699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c926d-f380-429e-bafb-b2b61c2b5fe8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:21.955322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 584/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:21.961416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:22.056879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60554be-1daf-4dee-b349-ec6bed6fe224 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:22.091810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:22.092513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:22.092801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:22.093084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:22.093802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:22.095243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2093f-590e-4da8-ab28-324123803402 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:22.118515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 554/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:41:21 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:24.454140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a53e4-12b7-49e0-acf5-c42088131f3e None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:24.454693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a53e4-12b7-49e0-acf5-c42088131f3e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:24.454693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a53e4-12b7-49e0-acf5-c42088131f3e None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:24.455971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 585/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:24.461778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:24.462193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:24.462299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:24.481035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:24.482208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:24.554502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 555/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:24.587202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a53e4-12b7-49e0-acf5-c42088131f3e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:24.697010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a53e4-12b7-49e0-acf5-c42088131f3e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:24.730914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:24.731630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:24.731906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:24.732178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:24.743412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:24.756312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:24.757492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:24.758451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 586/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:24.764417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:24.866426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cd4bd-255d-4ce9-a9f5-60d649a9b9b2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:24.902620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:24.903278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:24.903555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:24.903827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:24.904581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:24.904882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:24.906243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:24.914136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 556/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:24.921266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:25.050659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf36b4-9bbb-40e0-a88e-e84ca8d8ee93 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:25.090789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:25.091667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:25.091949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:25.092215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] PATH_INFO: `/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:25.106123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] RBAC: Authorizing `identity:get_user(user_id=fdc213b6e7f545f1935e6bf9d2c11bd4)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:25.107565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:25.109243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 587/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:24 2022] GET /identity/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4 => generated 286 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:25.115198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:25.219174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad7af6-b05a-4e90-947b-0c738ec2989a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:25.257179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:25.257891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:25.258165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:25.258445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:25.324622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:25.325993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:25.422104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 557/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:25.428478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:25.529383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760fa13-b881-4e87-bb5e-32367ebc2767 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:25.562372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:25.563255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:25.563529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:25.563800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] PATH_INFO: `/v3/projects/848424cd1a3f471fb42d7724d055bf71` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:25.564996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] RBAC: Authorizing `identity:get_project(project_id=848424cd1a3f471fb42d7724d055bf71)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:25.566427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cc22-cce4-474e-8ee3-bdd0081fa10f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:25.568144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 588/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:25 2022] GET /identity/v3/projects/848424cd1a3f471fb42d7724d055bf71 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:25.575336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:25.683770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b98cde-89bc-49b3-96bf-e46ee1dc072b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:25.717929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:25.718559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:25.718835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:25.719107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:25.719928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=848424cd1a3f471fb42d7724d055bf71, user.id=fdc213b6e7f545f1935e6bf9d2c11bd4) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:25.720241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:25.721695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c9c13-110c-45d5-bd86-ace7719836dd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:25.727247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 558/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:25 2022] GET /identity/v3/role_assignments?user.id=fdc213b6e7f545f1935e6bf9d2c11bd4&scope.project.id=848424cd1a3f471fb42d7724d055bf71&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:27.926420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b64f3-ca8b-4d7f-a93a-58e559cd34c4 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:27.927285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b64f3-ca8b-4d7f-a93a-58e559cd34c4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:27.927285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b64f3-ca8b-4d7f-a93a-58e559cd34c4 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:27.929522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 589/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:27.937669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:27.938109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:27.938221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:27.956693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:27.957868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:28.029755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 559/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:41:28.063277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b64f3-ca8b-4d7f-a93a-58e559cd34c4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:28.174857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b64f3-ca8b-4d7f-a93a-58e559cd34c4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:28.211571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:28.212390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:28.212674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:28.212943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:28.224421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:28.232107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:28.233190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:28.234515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 590/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:28.242783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:28.348670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b395813-2624-4024-8cfe-cf862f415f7c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:28.383519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:28.384175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:28.384491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:28.384766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:28.385513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:28.385817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:28.387198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:28.396271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 560/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:28.406219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:28.509952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a3451-41cb-429b-af7f-7dd5aaa09057 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:28.546657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:28.548013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:28.548390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:28.548772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] PATH_INFO: `/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:28.550909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] RBAC: Authorizing `identity:get_user(user_id=fdc213b6e7f545f1935e6bf9d2c11bd4)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:28.553354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:28.555749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 591/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:28 2022] GET /identity/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:28.562384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:28.674242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76d37d-5b3d-45a6-a9f6-4f4d818fc0ca None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:28.708549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:28.709408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:28.709687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:28.709960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] PATH_INFO: `/v3/projects/848424cd1a3f471fb42d7724d055bf71` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:28.711332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] RBAC: Authorizing `identity:get_project(project_id=848424cd1a3f471fb42d7724d055bf71)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:28.712955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b55b4-5297-47c7-a31d-4cd0f329674b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:28.714664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 561/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:28 2022] GET /identity/v3/projects/848424cd1a3f471fb42d7724d055bf71 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:28.724433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:28.843351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649a5f-e3ed-497d-bd3c-94a174e1371b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:28.878979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:28.879845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:28.880123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:28.880428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] PATH_INFO: `/v3/projects/848424cd1a3f471fb42d7724d055bf71/users/fdc213b6e7f545f1935e6bf9d2c11bd4/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:28.882487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=848424cd1a3f471fb42d7724d055bf71, user_id=fdc213b6e7f545f1935e6bf9d2c11bd4, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:28.884036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b82765-f3ae-4f6b-8566-03c5a4229bb8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:28.901731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 592/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:41:28 2022] PUT /identity/v3/projects/848424cd1a3f471fb42d7724d055bf71/users/fdc213b6e7f545f1935e6bf9d2c11bd4/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:41:31.079606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b6cd-1c45-44a4-a52f-13970b96ab96 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.080164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b6cd-1c45-44a4-a52f-13970b96ab96 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.080164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727b6cd-1c45-44a4-a52f-13970b96ab96 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.081549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 562/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:31.087504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.087948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.088038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.106520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:31.107696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:31.180767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 593/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:31.211299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727b6cd-1c45-44a4-a52f-13970b96ab96 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:31.318218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727b6cd-1c45-44a4-a52f-13970b96ab96 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:31.351185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:31.351921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.352360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.352654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.363240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:31.371191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:31.372374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:31.373399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 563/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:31.379737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:31.500647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838a62e-aca9-489e-9c6e-91c4bc0f5a5d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:31.537677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:31.537677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.537677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.537677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.538373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:31.538678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:31.540078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:31.549884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 594/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:31.556230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:31.654881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86499c-e58f-4611-b160-c978288bcce1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:31.688833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:31.689725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.690001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.690267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] PATH_INFO: `/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.691571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=fdc213b6e7f545f1935e6bf9d2c11bd4)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:31.693023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:31.694633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 564/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:31 2022] GET /identity/v3/users/fdc213b6e7f545f1935e6bf9d2c11bd4 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:31.701026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:31.816593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a9b8c-f560-4b3d-9699-714c1bc85148 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:31.850001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:31.850643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:31.850925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:31.851197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:31.922061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:31.923426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:32.019661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 595/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:32.029640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:32.133259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785385-ef76-4154-8fe5-1dddf0128ae5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:32.179033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:32.179835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:32.180415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:32.180415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] PATH_INFO: `/v3/projects/848424cd1a3f471fb42d7724d055bf71` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:32.181674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] RBAC: Authorizing `identity:get_project(project_id=848424cd1a3f471fb42d7724d055bf71)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:32.183111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fad78-34cc-44a4-ae92-459c57fefbbb None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:32.184871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 565/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:32 2022] GET /identity/v3/projects/848424cd1a3f471fb42d7724d055bf71 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:32.191266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:32.290511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ead5-3c77-4cc9-8a57-f389a9b0aff0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:32.325117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:32.325758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:32.326040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:32.326309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:32.327136 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=848424cd1a3f471fb42d7724d055bf71, user.id=fdc213b6e7f545f1935e6bf9d2c11bd4) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:32.327449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:32.328926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00820c4a-f378-4f14-b65d-84bd5a363bfd None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:32.334745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 596/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:32 2022] GET /identity/v3/role_assignments?user.id=fdc213b6e7f545f1935e6bf9d2c11bd4&scope.project.id=848424cd1a3f471fb42d7724d055bf71&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:34.543881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1807e5-c632-464e-9867-56f36038ddff None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:34.544804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1807e5-c632-464e-9867-56f36038ddff None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:34.544804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1807e5-c632-464e-9867-56f36038ddff None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:34.546927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 566/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:34.553468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:34.554228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:34.554383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:34.573272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:34.574493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:34.647886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 597/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:34.683087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1807e5-c632-464e-9867-56f36038ddff None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:34.797113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1807e5-c632-464e-9867-56f36038ddff None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:34.838421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:34.839063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:34.839344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:34.839619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:34.909587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:34.911043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:35.022234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 567/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:35.028717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:35.134876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a607ce8-11a7-430e-94be-a3ec541790c7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:35.176999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:35.177802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:35.178080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:35.178352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:35.185163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:35.186601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-18f4a2c3-e904-4b1b-894a-db460256afa8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:35.200683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:35.201855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:35.202722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 598/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:41:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:35.210445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:35.314513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbe6-bce5-4d5a-bc49-9ba4a74fdc82 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:35.348556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:35.349204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:35.349485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:35.349759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:35.350479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:35.350764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:35.352200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfec60-e1a9-4c03-80a9-f28f9af10258 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:35.359834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 568/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 12 00:41:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:37.554061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8510c-b112-419c-a897-ed88fd387f52 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:37.554957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8510c-b112-419c-a897-ed88fd387f52 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:37.554957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8510c-b112-419c-a897-ed88fd387f52 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:37.556703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 599/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:37.563075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90ce8-9710-45af-8996-7a5162610989 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:37.563523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90ce8-9710-45af-8996-7a5162610989 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:37.563630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90ce8-9710-45af-8996-7a5162610989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:37.589135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9db90ce8-9710-45af-8996-7a5162610989 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:37.590320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9db90ce8-9710-45af-8996-7a5162610989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:37.660923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 569/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:37.692020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8510c-b112-419c-a897-ed88fd387f52 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:37.798228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8510c-b112-419c-a897-ed88fd387f52 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:37.831642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:37.832212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:37.832518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:37.832789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:37.833518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:37.834877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6dd1-b87a-4c0e-b3cd-743bf25968ad None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:37.852933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 600/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 12 00:41:37 2022] POST /identity/v3/domains => generated 267 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:40.081407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352a1a-e34d-492f-92bf-b78bf971eef7 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:40.081997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352a1a-e34d-492f-92bf-b78bf971eef7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:40.081997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352a1a-e34d-492f-92bf-b78bf971eef7 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:40.083322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 570/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:40.088827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:40.089288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:40.089397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:40.109068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:40.110258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:40.186263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 601/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:40.209830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81352a1a-e34d-492f-92bf-b78bf971eef7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:40.311217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81352a1a-e34d-492f-92bf-b78bf971eef7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:40.345483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:40.346280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:40.346553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:40.346820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] PATH_INFO: `/v3/domains/4297cdb40f7346b795e27f3486ff79c3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:40.357474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4297cdb40f7346b795e27f3486ff79c3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:40.358920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0166bed-078c-4072-80e4-b742fc4ba5e6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:40.360565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 571/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 12 00:41:40 2022] GET /identity/v3/domains/4297cdb40f7346b795e27f3486ff79c3 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:40.366798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:40.460659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d38ba-79b7-45a1-b247-7f3cf957f821 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:40.495154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:40.495801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:40.496080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:40.496385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:40.497091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:40.498516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df59a5-8c81-40c1-a2be-105f585b0aef None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:40.530265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 602/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:41:40 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:42.758036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35d21-8c5c-4c44-b033-c00008f86b95 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:42.758935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35d21-8c5c-4c44-b033-c00008f86b95 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:42.758935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35d21-8c5c-4c44-b033-c00008f86b95 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:42.760604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 572/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:42.767192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:42.767620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:42.767787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:42.793764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:42.794943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:42.865245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 603/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:42.890279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d35d21-8c5c-4c44-b033-c00008f86b95 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:43.002094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d35d21-8c5c-4c44-b033-c00008f86b95 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:43.035774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:43.036596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:43.036874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:43.037142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] PATH_INFO: `/v3/domains/4297cdb40f7346b795e27f3486ff79c3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:43.038428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=4297cdb40f7346b795e27f3486ff79c3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:43.039888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081197c1-81bb-4c77-a9ad-cc5cb9d364aa None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:43.041513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 573/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 12 00:41:42 2022] GET /identity/v3/domains/4297cdb40f7346b795e27f3486ff79c3 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:43.048057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:43.149819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764f76b-7296-4708-b2bf-2cddffc35b69 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:43.183555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:43.184209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:43.184515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:43.184786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:43.185487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:43.186919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dff0f-72fb-47a9-938a-765d57bb2ced None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:43.210486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 604/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:41:43 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:41:45.453753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f141-5844-4ce7-af8e-1d9e51c3020b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:45.454601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f141-5844-4ce7-af8e-1d9e51c3020b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:45.454601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f141-5844-4ce7-af8e-1d9e51c3020b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:45.456758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 574/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:45.462262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:45.462689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:45.462803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:45.481513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:45.482667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:45.550954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 605/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:45.574758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f141-5844-4ce7-af8e-1d9e51c3020b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:45.698800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f141-5844-4ce7-af8e-1d9e51c3020b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:45.733020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:45.733744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:45.734025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:45.734300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:45.746282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:45.754122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:45.755184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:45.756415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 575/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:45.761720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:45.856055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f2765-7b65-4f32-bd30-a60718354a89 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:45.902955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:45.903609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:45.903889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:45.904152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:45.904927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:45.905231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:45.906591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:45.915213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 606/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:45.921277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:46.026326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c9bfb-86e1-4cb6-9d85-6b79c98419a3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:46.060071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:46.060961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:46.061238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:46.061504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] PATH_INFO: `/v3/users/70570543615a486292128798c9fb0dda` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:46.074813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] RBAC: Authorizing `identity:get_user(user_id=70570543615a486292128798c9fb0dda)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:46.076281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:46.078003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 576/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:45 2022] GET /identity/v3/users/70570543615a486292128798c9fb0dda => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:46.086960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:46.188554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b361a7-4d84-4d8b-a663-dcabd3454af4 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:46.221966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:46.222604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:46.222884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:46.223155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:46.289035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:46.290491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:46.386323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 607/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:46.395178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:46.493951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a872-f0fe-48f9-9269-27e316420e35 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:46.530660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:46.531513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:46.531788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:46.532061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] PATH_INFO: `/v3/projects/d13380dd0d6849a9aa330cd5270f47f0` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:46.533459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] RBAC: Authorizing `identity:get_project(project_id=d13380dd0d6849a9aa330cd5270f47f0)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:46.534893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd8ca1-dbbc-4637-b9dd-9c70b65339cc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:46.536598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 577/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:46 2022] GET /identity/v3/projects/d13380dd0d6849a9aa330cd5270f47f0 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:46.543235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:46.654394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfd36-c0fd-4a74-97d8-b690b26138f8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:46.689135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:46.689774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:46.690051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:46.690324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:46.691139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=d13380dd0d6849a9aa330cd5270f47f0, user.id=70570543615a486292128798c9fb0dda) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:46.691448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:46.692902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604ffc-49db-4882-af15-2c22f437abce None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:46.698503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 608/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:46 2022] GET /identity/v3/role_assignments?user.id=70570543615a486292128798c9fb0dda&scope.project.id=d13380dd0d6849a9aa330cd5270f47f0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:41:48.888947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd136d-96d3-4d1d-adc7-b46cb4eaa942 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:48.889543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd136d-96d3-4d1d-adc7-b46cb4eaa942 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:48.889543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd136d-96d3-4d1d-adc7-b46cb4eaa942 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:48.890889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 578/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:48.897359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:48.898059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:48.898250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:48.917512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:48.918690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:48.987817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 609/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:41:49.012987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd136d-96d3-4d1d-adc7-b46cb4eaa942 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:49.119484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd136d-96d3-4d1d-adc7-b46cb4eaa942 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:49.160403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:49.161495 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:49.161828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:49.162157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:49.179761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:49.187533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:49.188599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:41:49.197097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 579/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:49.197950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:49.307722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d08a1-a33c-4ff4-858a-d5b20046b9b2 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:49.342298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:49.342944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:49.343211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:49.343499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:49.344222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:49.344556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:49.345914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:49.353867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 610/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:49.363293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:49.475357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f38b-e6e5-4f3c-ab2d-ac760676a034 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:49.512616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:49.513476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:49.513749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:49.514014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] PATH_INFO: `/v3/users/70570543615a486292128798c9fb0dda` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:49.515240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] RBAC: Authorizing `identity:get_user(user_id=70570543615a486292128798c9fb0dda)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:49.516703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:49.518292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 580/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:49 2022] GET /identity/v3/users/70570543615a486292128798c9fb0dda => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:49.524790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:49.621704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e508-05b1-4d06-b910-2b1b169a7f76 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:49.654723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:49.655550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:49.655850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:49.656115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] PATH_INFO: `/v3/projects/d13380dd0d6849a9aa330cd5270f47f0` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:49.657320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] RBAC: Authorizing `identity:get_project(project_id=d13380dd0d6849a9aa330cd5270f47f0)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:49.658738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a6f7e-3880-4b06-b54f-6b6475b6f534 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:49.660461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 611/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:49 2022] GET /identity/v3/projects/d13380dd0d6849a9aa330cd5270f47f0 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:49.670949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:49.779058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdfe54-1308-4a08-b95f-422048e19200 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:49.812614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:49.813486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:49.813765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:49.814039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] PATH_INFO: `/v3/projects/d13380dd0d6849a9aa330cd5270f47f0/users/70570543615a486292128798c9fb0dda/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:49.816101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13380dd0d6849a9aa330cd5270f47f0, user_id=70570543615a486292128798c9fb0dda, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:49.817577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237f099-edb0-4318-b8b5-6c5b1bdd0658 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:49.834349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 581/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:41:49 2022] PUT /identity/v3/projects/d13380dd0d6849a9aa330cd5270f47f0/users/70570543615a486292128798c9fb0dda/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:41:51.994328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4007cd-df6d-4334-8623-e61c0ebf8dac None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:51.995224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4007cd-df6d-4334-8623-e61c0ebf8dac None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:51.995224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4007cd-df6d-4334-8623-e61c0ebf8dac None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:51.997450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 612/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:41:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:41:52.006840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:52.007268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:52.007452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:52.031993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:41:52.033213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:41:52.101279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 582/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:41:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:41:52.124544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4007cd-df6d-4334-8623-e61c0ebf8dac None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:52.246579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4007cd-df6d-4334-8623-e61c0ebf8dac None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:52.280143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:52.280960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:52.281241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:52.281510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:52.292492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:41:52.299952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:41:52.301030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:41:52.302545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 613/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:41:52.311127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:52.410473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d0fe3-bba9-443d-965c-d838bb2fdb02 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:52.444833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:52.445489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:52.445766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:52.446038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:52.446994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:52.447304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:52.448707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:52.456741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 583/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:52.466387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:52.567759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582436a5-71a9-4c0c-92e0-9f355f375d9f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:52.602314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:52.603386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:52.603661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:52.603931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] PATH_INFO: `/v3/users/70570543615a486292128798c9fb0dda` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:52.605256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] RBAC: Authorizing `identity:get_user(user_id=70570543615a486292128798c9fb0dda)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:52.606702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:52.608309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 614/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 12 00:41:52 2022] GET /identity/v3/users/70570543615a486292128798c9fb0dda => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:52.614874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:52.719788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ead6-7e57-46d5-8776-44079d07a3b5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:52.753002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:52.753630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:52.753905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:52.754178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:52.819000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:52.820392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:52.926438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 584/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:41:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:41:52.933680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:53.036623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f17ce-480e-482e-b070-541ef7acded7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:53.074616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:53.075415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:53.075693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:53.075962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] PATH_INFO: `/v3/projects/d13380dd0d6849a9aa330cd5270f47f0` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:53.077396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] RBAC: Authorizing `identity:get_project(project_id=d13380dd0d6849a9aa330cd5270f47f0)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:53.078866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdc99-acf3-4235-ae23-aaf354d56264 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:53.080615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 615/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 12 00:41:52 2022] GET /identity/v3/projects/d13380dd0d6849a9aa330cd5270f47f0 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:41:53.087763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:41:53.193298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85612dd-4865-448f-a462-38a3f99c9dc7 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:41:53.226691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:41:53.227326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:41:53.227626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:41:53.227909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:41:53.228779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=d13380dd0d6849a9aa330cd5270f47f0, user.id=70570543615a486292128798c9fb0dda) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:41:53.229089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:41:53.230526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ba6af-3064-4d04-9af7-eaab9e9c0a74 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:41:53.236886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 585/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:41:53 2022] GET /identity/v3/role_assignments?user.id=70570543615a486292128798c9fb0dda&scope.project.id=d13380dd0d6849a9aa330cd5270f47f0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:54.105139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a69c1-f7e7-48ac-b00a-b5f2163e89ec None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:54.105139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a69c1-f7e7-48ac-b00a-b5f2163e89ec None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:54.105139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a69c1-f7e7-48ac-b00a-b5f2163e89ec None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:54.105139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 616/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:42:54.113910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:54.114503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:54.114742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:54.134664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:42:54.136124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:42:54.220717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 586/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:42:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:42:54.247168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a69c1-f7e7-48ac-b00a-b5f2163e89ec None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:54.369791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a69c1-f7e7-48ac-b00a-b5f2163e89ec None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:54.409482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:54.410268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:54.410545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:54.410810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:54.422090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:54.423836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:42:54.441624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:54.442814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:54.443795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:42:54.443795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:42:54.443795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:54.443795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:42:54.445773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 617/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:42:54.451922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:54.572450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b0113-7b9d-4d6f-bc44-2fc73614a001 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:54.611617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:54.612369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:54.612627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:54.612898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:54.613635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:54.613935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:54.615357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45289ae-6988-4b92-85b7-dacf3032bed8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:54.632918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 587/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:54.634155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:54.750918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a316d-a835-4e4a-a9c5-9214c6eb375a None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:54.788990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:54.789733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:54.790048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:54.790336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:54.791187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:54.792891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c033f-4b6c-4138-a6a2-3363d27f8323 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:54.817571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 618/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 12 00:42:54 2022] POST /identity/v3/users => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:42:57.163237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a823538-6041-4608-8f8f-75c3ab096399 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.163818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a823538-6041-4608-8f8f-75c3ab096399 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.163818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a823538-6041-4608-8f8f-75c3ab096399 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.165399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 588/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:42:57.170772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.171381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.171381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.192024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:42:57.192024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:42:57.266762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 619/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:42:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:42:57.289438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a823538-6041-4608-8f8f-75c3ab096399 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:57.403207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a823538-6041-4608-8f8f-75c3ab096399 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:57.437800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:57.438603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.438924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.439202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.451402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:42:57.459443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:57.460469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:42:57.466727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 589/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:42:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:42:57.468852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:57.573059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555b26-ecd8-445c-b4ba-a45c170e92b1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:57.608273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:57.608954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.609286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.609533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.610414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:57.610707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:57.612030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:57.619985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 620/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:42:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:57.626479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:57.726015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3906fb-d941-4eae-96a8-9053d563d82d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:57.762827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:57.763908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.764238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.764585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.775409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:57.776933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:42:57.797536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:57.798594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:57.799491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:42:57.799491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:42:57.799491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:57.799491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:42:57.805866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 590/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 00:42:57.807038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:57.921707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350d561-68dd-475b-8806-c8b9f97b5b58 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:57.959574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:57.960274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:57.960565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:57.960821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:57.961545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:57.961900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:57.963234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:57.977563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 621/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:57.983397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:58.093363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9a64-ae75-4f67-927b-03d6620f7817 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:58.130457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:58.131289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:58.131570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:58.131843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:58.140876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:58.145009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:42:58.165046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:58.166770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:42:58.167751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 591/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:42:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:42:58.170026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:58.288201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a3266-46cb-4db7-9e95-99fc06833aee None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:58.331317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:58.332007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:58.332310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:58.332577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:58.333436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:58.333741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:58.335226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:58.358964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 622/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:42:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:58.365365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:58.478915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cdbf1-b318-4056-879e-a6540b9f3c90 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:58.523431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:58.524127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:58.524423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:58.524831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:58.604031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:58.605845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:58.726779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 592/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:42:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 12 00:42:58.733612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:58.863230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a575-de9d-4c43-80b3-db8afb4cd0d9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:58.908595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:58.909432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:58.909766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:58.910049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:58.923251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:58.924943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:42:58.947866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:58.948931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:58.949967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:42:58.949967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:42:58.949967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:42:58.949967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:42:58.949967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 623/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:42:58.956083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:59.077789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384eca-0c8b-4615-a283-87415343283f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:59.119311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:59.120400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:59.120884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:59.121346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:59.122210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:59.122583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:59.124458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:59.147906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 593/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:59.154568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:59.269966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aea19-9181-4cd8-b89c-52240deb56c7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:59.307048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:59.307925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:59.308134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:59.308439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:59.315582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:59.317089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:42:59.332950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:42:59.333986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:42:59.334909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 624/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:42:59.342112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:59.458718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7ba2-d45e-4f3a-9e4f-48749afe6314 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:59.509857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:59.510531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:59.510890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:59.511164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:59.512133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:59.512524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:59.514128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9845d11-8664-4b04-851a-67c7816c1726 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:59.530746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 594/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:42:59.539058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:42:59.649664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f882-4350-45ca-9b73-6dd2d5b1c667 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:42:59.684632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:42:59.685712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:42:59.686200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:42:59.686694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:42:59.687729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=b7dc2677d5c74bc7ad7770c3575f2b74) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:42:59.688265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:42:59.689908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724738-1384-4c25-85b1-88406fa5edb5 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:42:59.695632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 625/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:42:59 2022] GET /identity/v3/role_assignments?user.id=b7dc2677d5c74bc7ad7770c3575f2b74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:02.059474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615db38-4323-4f9a-8096-0e6262601a09 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.060043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615db38-4323-4f9a-8096-0e6262601a09 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.060043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615db38-4323-4f9a-8096-0e6262601a09 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.061426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 595/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:02.067268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.067741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.067834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.087004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:02.088241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:02.161348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 626/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:02.188671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615db38-4323-4f9a-8096-0e6262601a09 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:02.299683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615db38-4323-4f9a-8096-0e6262601a09 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:02.337592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:02.338820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.339386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.339859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.352694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:43:02.362798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:02.363913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:43:02.365006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:02.367374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 596/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:02.374186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:02.480219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30accb-0d14-40d0-b647-f6cc6a5e6df1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:02.523677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:02.524702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.525110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.525424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.526265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] RBAC: Adding query filter params (name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:02.526555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:02.528160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:02.538552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 627/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:02.545183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:02.657548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab57a3-7574-42ec-b54e-e60f438303f1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:02.691846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:02.692673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.692955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.693223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.704088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:02.705675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:02.724164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:02.725364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:02.726568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:02.726568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:02.726568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:02.726568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:02.726568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 597/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:02.731976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:02.838536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decda98-244c-4505-bd4b-3e86ac34a2ac None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:02.873985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:02.874658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:02.874942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:02.875218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:02.875947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:02.876268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:02.877656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:02.894443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 628/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:02.895749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.010806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333d209-4218-4b10-b188-e2110636a4a3 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.047407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.048228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.048541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.048817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.056343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.057796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:43:03.073603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:03.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:03.076348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 598/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:03.080967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.183331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96c14-20dd-4b3e-9d6d-4a50ebb9cc66 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.218555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.219305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.219657 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.219935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.220930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:03.221252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.222995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.242330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 629/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:43:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:03.249581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.356585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86e86-5d84-4b8d-a3b4-bded780b2c24 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.396510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.397311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.397590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.397862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.409393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.410907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:03.427875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:03.429119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:03.430548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:03.430548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:03.430548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:03.430548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:03.430548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 599/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:03.435663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.539803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66903-c5fc-4394-b75f-d3ad54b1b272 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.577509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.578252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.578516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.578842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.579568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:03.579876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.581369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.593950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 630/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:03.599698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.704118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582195-5ae8-4861-97c7-afbbe9284bb5 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.737820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.738613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.738895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.739170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.745715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.747858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:43:03.763150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:03.764341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:03.765356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 600/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:03.770465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:03.871634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dd34a-5746-49a3-9aba-6aaf216d58b2 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:03.904795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:03.905615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:03.905897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:03.906168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:03.906949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:03.907254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:03.908732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273aa30-d2fb-4f7e-a62c-41535bd6de37 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:03.921762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 631/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:03.929111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:04.031739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d765c-e983-4648-a511-f6e5a1c0d67b None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:04.069010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:04.069924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:04.070191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:04.070465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/b7dc2677d5c74bc7ad7770c3575f2b74/roles/38af56321a604498b2a77ba5c65b2ad5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:04.088639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=b7dc2677d5c74bc7ad7770c3575f2b74, role_id=38af56321a604498b2a77ba5c65b2ad5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:04.090315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c13a0-8322-4b11-ab99-0c48c6fe114f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:04.104422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 601/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:43:03 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/b7dc2677d5c74bc7ad7770c3575f2b74/roles/38af56321a604498b2a77ba5c65b2ad5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:43:06.511163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4790c8-7ed0-4ede-84f1-4bb46987621c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:06.511708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4790c8-7ed0-4ede-84f1-4bb46987621c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:06.511708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4790c8-7ed0-4ede-84f1-4bb46987621c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:06.513201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 632/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:06.518586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:06.518988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:06.519142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:06.536010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:06.537232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:06.604717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 602/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:06.628794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4790c8-7ed0-4ede-84f1-4bb46987621c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:06.734038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4790c8-7ed0-4ede-84f1-4bb46987621c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:06.771933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:06.772795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:06.773143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:06.773431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] PATH_INFO: `/v3/roles/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:06.785273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:43:06.794484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:06.795698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:06.796773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 633/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 12 00:43:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:06.802005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:06.907484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd44b8-16f0-4d4f-a881-378de20e94ba None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:06.943953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:06.944802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:06.945082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:06.945350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:06.946206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:06.946426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:06.948051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:06.958615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 603/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 12 00:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:06.965375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:07.071355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce601edd-b6bc-4316-b89a-d38f5521acca None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:07.106888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:07.107932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:07.108377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:07.108707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:07.120630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:07.122385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:07.140503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:07.141681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:07.147710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:07.147710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:07.147710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:07.147710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:07.147710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 634/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:07.148603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:07.269596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3a16b-2f67-4143-8fd4-ab0eab1303a1 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:07.306331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:07.306967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:07.307248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:07.307523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:07.308282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:07.308584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:07.310183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:07.322598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 604/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:43:07.328588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:07.438953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e8e7-ecf8-4403-802f-a17211e96b7c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:07.478507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:07.479333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:07.479701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:07.480006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] PATH_INFO: `/v3/users/placement` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:07.489916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:07.491385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:43:07.507462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:07.508720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:07.509912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 635/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:07.515369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:07.630515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef76c-274b-40c7-8c4a-e9ee4c9f4930 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:07.667711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:07.668587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:07.668879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:07.669233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:07.670249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:07.670561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:07.672012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:07.691241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 605/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:43:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:07.697375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:07.804885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f069-3a94-42fd-9887-dfc822b6343b None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:07.841564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:07.842150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:07.842453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:07.842732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:07.918417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:07.920489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.024542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 636/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:43:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:43:08.031222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:08.148491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd584-93b0-4ff1-8e0a-ae6173b86246 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:08.189728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:08.190751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:08.191103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:08.191376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:08.202310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:08.203903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:08.222868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:08.224509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:08.226056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:08.226056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:08.226056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:08.226056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:08.226056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 606/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:08.230385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:08.337698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0389b1-2827-40e1-832e-a1b5766600b1 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:08.377122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:08.377777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:08.378055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:08.378326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:08.379043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:08.379347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:08.383817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.393975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 637/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:08.399709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:08.512973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7b5d-1a22-47a4-9947-b077c3bdbe45 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:08.549963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:08.550755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:08.551034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:08.551305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:08.558028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:08.559482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:43:08.574849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:08.576062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:08.577006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:08.577436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 607/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:08.583399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:08.690129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e77d1f-4b73-4989-9108-176bb2363460 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:08.724418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:08.725127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:08.725350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:08.725625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:08.726419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:08.726760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:08.728140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07338fa0-8def-4993-ae82-dd52535b8aee None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.740857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 638/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:08.747515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:08.859068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5406-94e0-4aa4-8324-301ca459b80f None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:08.896269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:08.896918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:08.897195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:08.897464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:08.898289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] RBAC: Adding query filter params (role.id=38af56321a604498b2a77ba5c65b2ad5, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=b7dc2677d5c74bc7ad7770c3575f2b74) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:08.898595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:08.900054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc1ff9-298d-4ca2-b75e-318fd6ab405e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:08.905848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 608/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:43:08 2022] GET /identity/v3/role_assignments?user.id=b7dc2677d5c74bc7ad7770c3575f2b74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=38af56321a604498b2a77ba5c65b2ad5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:11.401709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002482-ac1d-48d3-97f7-3f3bc8a18ea0 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:11.402357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002482-ac1d-48d3-97f7-3f3bc8a18ea0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:11.402357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002482-ac1d-48d3-97f7-3f3bc8a18ea0 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:11.404655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 639/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:11.410694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:11.411234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:11.411349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:11.430410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:11.431694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:11.510156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 609/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:11.534139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002482-ac1d-48d3-97f7-3f3bc8a18ea0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:11.648752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002482-ac1d-48d3-97f7-3f3bc8a18ea0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:11.694039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:11.695076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:11.695456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:11.695855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:11.709701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:43:11.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:11.721163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:11.722200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 640/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:11.728826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:11.837255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d12f5-5f38-410c-b922-3b15c862b213 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:11.874934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:11.875712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:11.876002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:11.876302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:11.877076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:11.877431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:11.878967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:11.888999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 610/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:11.895631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:12.014131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571fbb-6da6-4629-b3f7-f0bb5df94781 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:12.051579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:12.052450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:12.052936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:12.053152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:12.065011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:12.066500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:12.085305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:12.086409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:12.087750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:12.087750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:12.087750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:12.087750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:12.087750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 641/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:12.093371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:12.215638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfb8c2-959f-4a8e-86ba-7e36963b2713 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:12.256981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:12.257652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:12.257954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:12.258810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:12.258977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:12.259314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:12.260891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:12.275455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 611/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:12.282755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:12.392805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c5286-e64b-4cbd-9696-af203c4fce22 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:12.429501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:12.430326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:12.430608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:12.430882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:12.438667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:12.440129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:43:12.454786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:12.455907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:12.456889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 642/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:12.462273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:12.574789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d69216-9026-4b91-b9d8-adf0cb4d33cf None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:12.611179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:12.611979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:12.612289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:12.612597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:12.613560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:12.613913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:12.615373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:12.634810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 612/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:43:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:12.641607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:12.744574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ca3c-f34f-45b1-a76d-026320e3caf7 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:12.780647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:12.781487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:12.781772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:12.782047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:12.854257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:12.856124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:12.963431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 643/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:43:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:43:12.969771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:13.079182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49441-9742-4b70-bcd3-cdf5a1ad9833 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:13.113957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:13.114704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:13.114984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:13.115256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:13.126302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:13.127994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:13.146785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:13.147822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:13.149313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:13.149313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:13.149313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:13.149313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:13.149313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 613/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:13.155197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:13.278402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa6aff-be6a-4a7b-8ac2-22f909c95ffc None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:13.316904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:13.317809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:13.318166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:13.318416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:13.319344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:13.319706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:13.321642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:13.334383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 644/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:13.340474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:13.442000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccc551-4dbe-48f8-9707-ac3fc020ece6 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:13.477440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:13.478376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:13.478666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:13.478943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:13.487871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:13.489351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:43:13.503298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:13.504401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:13.505771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 614/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:13.510516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:13.620399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b2de0-ab0a-413b-8dbd-7f9b8f48b9f9 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:13.657621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:13.658290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:13.658573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:13.658849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:13.659639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:13.659949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:13.661438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194ad6-70fd-49b4-93de-8f58ca22d8e3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:13.679239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 645/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:13.680742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:13.784428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb065-6351-4077-911b-04f8e1613538 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:13.821960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:13.822703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:13.822976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:13.823220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:13.824084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=b7dc2677d5c74bc7ad7770c3575f2b74) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:13.824524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:13.826028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cf951-c128-45c7-8305-b104dcf882b5 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:13.832713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 615/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:43:13 2022] GET /identity/v3/role_assignments?user.id=b7dc2677d5c74bc7ad7770c3575f2b74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:16.092882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bea4b-1cf7-485e-9cef-686bff3098c9 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.093772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bea4b-1cf7-485e-9cef-686bff3098c9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.093772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bea4b-1cf7-485e-9cef-686bff3098c9 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.095964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 646/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:16.101406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.101823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.101915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.120028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:16.121264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:16.202448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 616/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:16.226054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bea4b-1cf7-485e-9cef-686bff3098c9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:16.339736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bea4b-1cf7-485e-9cef-686bff3098c9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:16.374426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:16.375212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.375487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.375754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.385964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:43:16.393705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:16.394897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:16.395779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 647/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:43:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:16.401303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:16.503221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca510-ec08-4426-95bc-cedfe277ae63 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:16.540051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:16.540737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.541023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.541296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.542043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] RBAC: Adding query filter params (name=admin) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:16.542352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:16.544011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:16.552003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 617/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:43:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:16.558129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:16.660437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1046-7716-4591-8d9f-6c52ed7179f0 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:16.695001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:16.695781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.696060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.696370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.707423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:16.708903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:16.725902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:16.726894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:16.728634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:16.728634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:16.728634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:16.728634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:16.728634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 648/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:16.736084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:16.844297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5cc0-ac3f-4367-8b42-951d07abf11d None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:16.879840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:16.880511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:16.880799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:16.881072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:16.881795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:16.882101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:16.883477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:16.896966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 618/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:16.903481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.005341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22357540-8d3d-4d01-99d4-3932212a6132 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.044359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.045161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.045439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.045717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] PATH_INFO: `/v3/users/placement` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.053186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.054619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:43:17.069097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:17.070080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:17.071086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 649/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:17.078583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.195185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda6a30-95e1-49e4-84a6-d406ac5f0bfc None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.231255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.231922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.232200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.232507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.233347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:17.233658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.235110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.252539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 619/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:43:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:17.258620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.360642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741ad38-506a-4d68-9273-dbe8eeb1b57f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.395129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.395910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.396190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.396488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.407682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.409166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:17.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:17.427532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:17.428700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:17.428700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:17.428700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:17.428700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:17.428700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 650/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:17.433482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.544888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f01d-87b6-49f0-b869-229038d6810a None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.590855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.591762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.592073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.592471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.593525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:17.593883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.595956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.611703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 620/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:17.617754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.748641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6af8a-b429-47ce-9dbb-e428df225b92 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.784587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.785374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.785655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.785935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.792946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.794399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:43:17.808589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:17.809589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:17.810601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 651/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:43:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:17.817791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:17.920650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7f2ec-25fc-4544-bae9-ea0a1330e867 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:17.955926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:17.956602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:17.956880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:17.957148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:17.957923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:17.958224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:17.959664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6177ed-64e3-46e8-8fa4-16c1a61d4e7f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:17.971669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 621/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:17.981085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:18.104133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0311cef-aab3-4e59-a7fe-f424aaa3023c None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:18.147642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:18.148382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:18.148739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:18.149110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] PATH_INFO: `/v3/projects/3d005aee3baa40c2af159f881af910d0/users/b7dc2677d5c74bc7ad7770c3575f2b74/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:18.152791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] RBAC: Authorizing `identity:create_grant(project_id=3d005aee3baa40c2af159f881af910d0, user_id=b7dc2677d5c74bc7ad7770c3575f2b74, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:18.154659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fef9e4-333a-415e-b387-c3ae5b18c9ba None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:18.171367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 652/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 12 00:43:17 2022] PUT /identity/v3/projects/3d005aee3baa40c2af159f881af910d0/users/b7dc2677d5c74bc7ad7770c3575f2b74/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:43:20.351458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6b46-25c3-45ce-8d26-64cd0d3b8117 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:20.352376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6b46-25c3-45ce-8d26-64cd0d3b8117 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:20.352376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6b46-25c3-45ce-8d26-64cd0d3b8117 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:20.354544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 622/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:20.360843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:20.361563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:20.361723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:20.383191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:20.384402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:20.453450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 653/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:43:20.481381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc6b46-25c3-45ce-8d26-64cd0d3b8117 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:20.607649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc6b46-25c3-45ce-8d26-64cd0d3b8117 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:20.642335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:20.643094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:20.643397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:20.643661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:20.654248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 00:43:20.661706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:20.662752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:20.664183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 623/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 12 00:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:20.671572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:20.786072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d0ad-33e2-445d-b8b5-851cf799d4f0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:20.824042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:20.824769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:20.825044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:20.825313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:20.826043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:20.826343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:20.827701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:20.835567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 654/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 12 00:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:20.842394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:20.945781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891cfa-d247-4a76-825f-e78c5e57a972 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:20.980483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:20.981775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:20.982054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:20.982328 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:20.992958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:20.994407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:21.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:21.014210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:21.015674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:21.015674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:21.015674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:21.015674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:21.015674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 624/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:21.023328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:21.133244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4875-1a0f-4614-8452-619f0515c7e8 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:21.178966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:21.180012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:21.180496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:21.181161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:21.182459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:21.182944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:21.185198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:21.201807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 655/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:21.208092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:21.323544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9327e-c235-442e-bfdd-0a61f7a9f00e None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:21.357441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:21.358239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:21.358516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:21.358793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:21.366441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:21.367883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 00:43:21.382503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:21.383593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:21.384776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 625/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:21.390562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:21.503198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de8268-8d87-4ebb-b7ee-ff1d7afab7e3 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:21.537222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:21.537867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:21.538144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:21.538412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:21.539257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:21.539578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:21.541026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:21.557673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 656/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 12 00:43:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:21.567515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:21.669033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e874e-3970-4ede-98dc-220777d1f3dd None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:21.702945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:21.703578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:21.703867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:21.704137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:21.773572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:21.775049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:21.871659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 626/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 12 00:43:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 00:43:21.878387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:21.978707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e01151-60db-4de0-89f1-b030fdff9c85 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:22.012954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:22.013737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:22.014018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:22.014293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:22.024465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:22.025908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 00:43:22.043410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:22.044461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:22.045388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 00:43:22.045388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 00:43:22.045388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 00:43:22.045388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:22.045388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 657/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:22.050610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:22.150659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f6cef-8ccc-4182-9f59-03d543ae36d8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:22.190512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:22.191178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:22.191461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:22.191740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:22.192495 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:22.192799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:22.194213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:22.205909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 627/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 12 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:22.211946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:22.316163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982877a5-8683-4ad6-a7c2-b4c423669350 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:22.352391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:22.353169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:22.353448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:22.353720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:22.360897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:22.362336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 00:43:22.376485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application with Lock( May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:22.377484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: ERROR keystone.server.flask.application May 12 00:43:22.378804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 658/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 12 00:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:22.383770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:22.484048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c56d2-d299-4753-a16b-bfb5950908f8 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:22.517958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:22.518606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:22.518888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:22.519159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:22.519943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:22.520288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:22.521729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed11f2-8ade-41cf-af73-b03afd3e514f None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:22.535262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 628/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 12 00:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:22.541894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:22.647360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656a32-b07d-4911-8143-9732bc652f13 None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:22.681446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:22.682082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:22.682364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:22.682635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:22.683609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] RBAC: Adding query filter params (role.id=33906b6d96ba49e6a74f107c852f7119, scope.project.id=3d005aee3baa40c2af159f881af910d0, user.id=b7dc2677d5c74bc7ad7770c3575f2b74) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:22.683955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:22.685446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df8c91-0bd0-42d5-8131-6ce74ac051f6 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:22.691222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 659/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 12 00:43:22 2022] GET /identity/v3/role_assignments?user.id=b7dc2677d5c74bc7ad7770c3575f2b74&scope.project.id=3d005aee3baa40c2af159f881af910d0&role.id=33906b6d96ba49e6a74f107c852f7119 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:24.905219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17bdb-68c0-4cff-89ab-63db36aaa7d4 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:24.906114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17bdb-68c0-4cff-89ab-63db36aaa7d4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:24.906114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17bdb-68c0-4cff-89ab-63db36aaa7d4 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:24.908308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 629/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:24.916280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:24.916713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:24.916824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:24.935990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:24.937182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:25.002818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 660/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:43:25.035762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17bdb-68c0-4cff-89ab-63db36aaa7d4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:25.147545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17bdb-68c0-4cff-89ab-63db36aaa7d4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:25.189370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:25.190119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:25.190399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:25.190663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:25.191356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:25.192775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:25.204684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:25.206197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:25.207269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 630/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:43:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:25.212556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:25.323675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18893a0c-8eeb-4822-be48-29994c719348 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:25.359265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:25.359923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:25.360200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:25.360500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:25.361203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] RBAC: Adding query filter params (name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:25.361512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:25.362886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e02c3-c0e8-4c93-91fd-ad42ff77acec None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:25.367741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 661/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 12 00:43:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:25.374599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:25.479260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6adee-ecba-48e6-b984-05328da6cd16 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:25.515380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:25.516024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:25.516311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:25.516579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] PATH_INFO: `/v3/services` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:25.517309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] RBAC: Adding query filter params (type=placement) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:25.517616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:25.518989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983befa-836b-46cf-aa36-1b60996d919e None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:25.523408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 631/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 12 00:43:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:27.679272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851c43-c57a-4824-8d0e-ede8f20ad796 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:27.680138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851c43-c57a-4824-8d0e-ede8f20ad796 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:27.680138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851c43-c57a-4824-8d0e-ede8f20ad796 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:27.682351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 662/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:27.688827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959b-c229-4f65-9853-99032e61ad5b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:27.689564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959b-c229-4f65-9853-99032e61ad5b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:27.689658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959b-c229-4f65-9853-99032e61ad5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:27.716770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a486959b-c229-4f65-9853-99032e61ad5b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:27.717960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a486959b-c229-4f65-9853-99032e61ad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:27.787481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 632/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:27.818260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63851c43-c57a-4824-8d0e-ede8f20ad796 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:27.920860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63851c43-c57a-4824-8d0e-ede8f20ad796 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:27.954732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764e85-6219-4b69-a219-8c978129980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:27.955313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764e85-6219-4b69-a219-8c978129980b None admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:27.955588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764e85-6219-4b69-a219-8c978129980b None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:27.955853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764e85-6219-4b69-a219-8c978129980b None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:27.956608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764e85-6219-4b69-a219-8c978129980b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:27.957963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764e85-6219-4b69-a219-8c978129980b None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:27.974974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 663/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 12 00:43:27 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:43:30.176762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe054f9-4359-4f55-9d61-d97d8777ddf6 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.177386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe054f9-4359-4f55-9d61-d97d8777ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.177386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe054f9-4359-4f55-9d61-d97d8777ddf6 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.178912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 633/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:30.185813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.186243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.186404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.209493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:30.211102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:30.286743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 664/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 00:43:30.311457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe054f9-4359-4f55-9d61-d97d8777ddf6 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:30.411800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe054f9-4359-4f55-9d61-d97d8777ddf6 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:30.444995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:30.445776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.446051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.446324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] PATH_INFO: `/v3/services/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.446967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:30.448317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:30.460024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:30.461166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:30.462345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 634/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:43:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:30.468199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:30.580625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab3302-6ba3-4015-9932-ffc4224346a4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:30.623983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:30.624994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.625461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.625843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.626935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] RBAC: Adding query filter params (name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:30.627395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:30.629606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:30.637551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 665/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 12 00:43:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:43:30.647146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:30.777019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900aba9f-c9e2-41d9-a11f-7dc26a3f8011 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:30.814900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:30.815542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.815825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.816100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.816923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] RBAC: Adding query filter params (interface=public, service_id=9152ab6fee53443681ea26e0618db783, region_id=RegionOne) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:30.817230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:30.818612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcc7e7-3be3-4de4-a650-3c1d293b0925 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:30.823704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 635/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 12 00:43:30 2022] GET /identity/v3/endpoints?service_id=9152ab6fee53443681ea26e0618db783&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:30.832662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:30.952161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc929c2-df92-4f70-a4ac-8345100bc72f None admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:30.986198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:30.986846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:30.987122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:30.987392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:30.988066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:30.989495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe5c01-9f87-4f73-af2f-510249bf8025 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:30.996592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 666/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 12 00:43:30 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:43:33.182813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626d54-71e0-4d9e-88b4-72122512125f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:33.183329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626d54-71e0-4d9e-88b4-72122512125f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:33.183329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626d54-71e0-4d9e-88b4-72122512125f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:33.184992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 636/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:43:33.193736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:33.194206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:33.194298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:33.217917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:43:33.219098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:43:33.290906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 667/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:43:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 00:43:33.314020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e626d54-71e0-4d9e-88b4-72122512125f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:33.435046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e626d54-71e0-4d9e-88b4-72122512125f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:33.469919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453b139-5270-414d-92ac-155119789040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:33.470713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b139-5270-414d-92ac-155119789040 None admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:33.470991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b139-5270-414d-92ac-155119789040 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:33.471258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b139-5270-414d-92ac-155119789040 None admin] PATH_INFO: `/v3/services/placement` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:33.471903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453b139-5270-414d-92ac-155119789040 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:33.473302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453b139-5270-414d-92ac-155119789040 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-f453b139-5270-414d-92ac-155119789040 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 00:43:33.485261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 00:43:33.486403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 00:43:33.488176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 637/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 12 00:43:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 00:43:33.496196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:33.607395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707125d-6f0f-4129-9365-85e30892fbe6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:33.641793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:33.642440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:33.642906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:33.643164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] PATH_INFO: `/v3/services` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:33.643889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:43:33.644192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:33.645594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29df19-7332-4189-8cdf-0fb7bb3686d2 None admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:33.650670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 668/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 12 00:43:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:43:33.656896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453b139-5270-414d-92ac-155119789040 None admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:43:33.761208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453b139-5270-414d-92ac-155119789040 None admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:43:33.796171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:43:33.796839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:43:33.797120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:43:33.797394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:43:33.798124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:43:33.799487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23052d6-56e7-4893-b57e-307ca46658d0 None admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:43:33.822453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 638/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 12 00:43:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:44:38.405952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2007082-62f1-4668-8fdf-9c732c6c2728 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:38.406534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2007082-62f1-4668-8fdf-9c732c6c2728 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:38.406534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2007082-62f1-4668-8fdf-9c732c6c2728 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:38.407798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 669/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:38.413171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971662c-3cd0-49b9-bae3-a1455dcfd88a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:38.413613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971662c-3cd0-49b9-bae3-a1455dcfd88a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:38.413701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971662c-3cd0-49b9-bae3-a1455dcfd88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:38.493760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3971662c-3cd0-49b9-bae3-a1455dcfd88a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:38.495267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3971662c-3cd0-49b9-bae3-a1455dcfd88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:38.671811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 639/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:38.744754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:38.745048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:38.745325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:38.747508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 670/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 12 00:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:38.755510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60020-bdd9-4157-9f9d-2ef991bc4bc0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:38.755804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60020-bdd9-4157-9f9d-2ef991bc4bc0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:38.756086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60020-bdd9-4157-9f9d-2ef991bc4bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:38.910784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-76b60020-bdd9-4157-9f9d-2ef991bc4bc0 None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:38.912155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-76b60020-bdd9-4157-9f9d-2ef991bc4bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:39.037139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 640/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 12 00:44:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:39.046450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:39.048796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:39.178385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a0a5-6d61-4cc5-9b0c-5b71fb4dae29 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:39.254941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:39.256012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:39.256544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:39.257017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:39.259584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:39.370237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:39.372795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dec589-5225-45cb-b80d-90672eaeef08 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:39.393005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 671/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:41.930034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:41.930479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:41.930754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:41.932038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 641/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:41.937574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db28f-4928-4e50-9cf9-baf313bde84f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:41.938237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db28f-4928-4e50-9cf9-baf313bde84f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:41.938929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db28f-4928-4e50-9cf9-baf313bde84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:42.052660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-282db28f-4928-4e50-9cf9-baf313bde84f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:42.054055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-282db28f-4928-4e50-9cf9-baf313bde84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:42.152723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 672/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:42.190215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:42.191837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:42.300338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f65e-70c7-4b8b-9408-39bada11c94a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:42.374201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:42.374842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:42.375125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:42.375402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:42.377045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:42.483390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:42.484771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0c767-62fb-4064-bbc3-679baf2b57b8 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:42.503257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 642/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:45.060365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:45.060942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:45.060942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:45.062311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 673/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:45.068719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c39e8-bb04-4ed8-b363-9f15c3b0a915 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:45.069160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c39e8-bb04-4ed8-b363-9f15c3b0a915 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:45.069332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c39e8-bb04-4ed8-b363-9f15c3b0a915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:45.145020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-572c39e8-bb04-4ed8-b363-9f15c3b0a915 None None] MFA Rules not processed for user `5deadb4ca8544f08a5e703d8a595cc62`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:45.146580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-572c39e8-bb04-4ed8-b363-9f15c3b0a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:45.242666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 643/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:45.296758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:45.298388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:45.402223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a785b-4b5a-449c-945d-926cacbb2b42 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:45.477870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:45.478661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:45.479151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:45.479424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:45.481092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:45.600998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:45.602444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74df68-c279-4747-96a9-b53631891806 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:45.622016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 674/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:48.437483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:48.438362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:48.438362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:48.440443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 644/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:48.450113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712d1ee-f96b-4f56-8feb-4f15b8dde690 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:48.450574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712d1ee-f96b-4f56-8feb-4f15b8dde690 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:48.450673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712d1ee-f96b-4f56-8feb-4f15b8dde690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:48.524520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a712d1ee-f96b-4f56-8feb-4f15b8dde690 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:48.525758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a712d1ee-f96b-4f56-8feb-4f15b8dde690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:48.625822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 675/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:48.669093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:48.671537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:48.786167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62e73-ffb6-4c04-895c-6f055b3bebfb None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:48.858139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:48.858804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:48.859082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:48.859363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:48.861067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:48.969046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:48.970481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaad43-9c69-4f51-8e82-b8d21298542b service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:48.988230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 645/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:51.844137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:51.844896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:51.844896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:51.846149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 676/1321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:51.851631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee378-353c-4e75-beff-bea8489818b6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:51.852103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee378-353c-4e75-beff-bea8489818b6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:51.852211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee378-353c-4e75-beff-bea8489818b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:51.918579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bd3ee378-353c-4e75-beff-bea8489818b6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:51.919756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bd3ee378-353c-4e75-beff-bea8489818b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:52.016152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 646/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:52.051734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:52.053288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:52.157811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52885b2d-0f09-4dd4-a91c-e8e208d87fdd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:52.234104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:52.234767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:52.235047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:52.235324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:52.237024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:52.349976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:52.351424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef358b-c3ba-47e6-8f89-eea3645db6e3 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:52.369869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 677/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:54.661436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:54.662029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:54.662029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:54.663348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 647/1324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:54.669127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb7feb-60c3-49cc-9af0-6320727511c3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:54.669592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb7feb-60c3-49cc-9af0-6320727511c3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:54.669799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb7feb-60c3-49cc-9af0-6320727511c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:54.733195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0dfb7feb-60c3-49cc-9af0-6320727511c3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:54.734467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0dfb7feb-60c3-49cc-9af0-6320727511c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:54.831267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 678/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:54.867861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:54.870180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:54.976820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432bf7c-2dbc-4f18-88f7-62f2aff3ab79 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:55.053465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:55.054118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:55.054393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:55.054661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:55.056283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:55.167122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:55.169257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e01929-c64c-40c3-98e4-285cfd982b7a service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:55.188806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 648/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:44:57.467428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:57.467961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:57.467961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:57.469306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 679/1327] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:44:57.474778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b06e2-9313-407f-a21b-6d15876d7e1d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:57.475228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b06e2-9313-407f-a21b-6d15876d7e1d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:57.475452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b06e2-9313-407f-a21b-6d15876d7e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:57.543292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-671b06e2-9313-407f-a21b-6d15876d7e1d None None] MFA Rules not processed for user `5deadb4ca8544f08a5e703d8a595cc62`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:44:57.544553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-671b06e2-9313-407f-a21b-6d15876d7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:57.642901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 649/1328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:44:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:44:57.680844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:44:57.682743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:57.786348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b52a3-10ff-4077-ba11-541a6085cebf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:44:57.861955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:44:57.862603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:44:57.862883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:44:57.863151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:44:57.864940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:44:57.974005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:44:57.975458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346ffbf-9d7c-4303-be51-01bbfec648b1 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:44:57.993329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 680/1329] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:44:57 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:00.511047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:00.511675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:00.511675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:00.513107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 650/1330] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:00.519297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931670c6-c5d6-4ac5-ad8a-bc15ff823b26 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:00.519741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931670c6-c5d6-4ac5-ad8a-bc15ff823b26 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:00.519741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931670c6-c5d6-4ac5-ad8a-bc15ff823b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:00.592456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-931670c6-c5d6-4ac5-ad8a-bc15ff823b26 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:00.593641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-931670c6-c5d6-4ac5-ad8a-bc15ff823b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:00.690096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 681/1331] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:00.725006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:00.726551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:00.830425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c11d9-d014-45c7-895d-7b281c55a1b2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:00.912093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:00.912886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:00.913165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:00.913439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:00.915091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:01.026915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:01.028419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11cb97-cebf-4a1c-a9e4-425a334b936a service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:01.046763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 651/1332] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:45:03.416324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:03.417162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:03.417162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:03.419223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 682/1333] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:03.425524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0c476-a500-4e38-be43-ec95b06eb8ed None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:03.426309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0c476-a500-4e38-be43-ec95b06eb8ed None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:03.426483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0c476-a500-4e38-be43-ec95b06eb8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:03.502426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-88c0c476-a500-4e38-be43-ec95b06eb8ed None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:03.503628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-88c0c476-a500-4e38-be43-ec95b06eb8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:03.606736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 652/1334] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:03.650993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:03.652675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:03.755036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28434719-b3e2-44c2-bebe-1fc344b7d470 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:03.837850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:03.838936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:03.839408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:03.839715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:03.841434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:03.954705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:03.956128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ef9c-c9d7-4872-bb7d-7922e05343a9 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:03.973703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 683/1335] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:06.487759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:06.488791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:06.488791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:06.490801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 653/1336] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:06.497475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba597528-8324-46e7-a085-21ffa85ee58b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:06.497946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba597528-8324-46e7-a085-21ffa85ee58b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:06.498047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba597528-8324-46e7-a085-21ffa85ee58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:06.567674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ba597528-8324-46e7-a085-21ffa85ee58b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:06.568900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ba597528-8324-46e7-a085-21ffa85ee58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:06.665050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 684/1337] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:06.708657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:06.711049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:06.817881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d9a3b-d598-4a6c-9e43-6295b6de0e20 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:06.891247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:06.891894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:06.892136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:06.892472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:06.894118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:07.001872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:07.003311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f88da-84e8-4e05-a976-d9be6ca85d48 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:07.021278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 654/1338] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:11.242566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff757d-5148-405e-8dc1-8cf2e5f97366 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:11.243131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff757d-5148-405e-8dc1-8cf2e5f97366 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:11.243131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff757d-5148-405e-8dc1-8cf2e5f97366 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:11.244581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 685/1339] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:11.250861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5632b3-b04e-4842-bcf7-c8cc8aad5be9 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:11.251336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5632b3-b04e-4842-bcf7-c8cc8aad5be9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:11.251437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5632b3-b04e-4842-bcf7-c8cc8aad5be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:11.323194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8d5632b3-b04e-4842-bcf7-c8cc8aad5be9 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:11.324344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8d5632b3-b04e-4842-bcf7-c8cc8aad5be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:11.434052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 655/1340] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:11.487493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04a0a-259f-4c9e-9a8d-40e281c5c955 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:11.487736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04a0a-259f-4c9e-9a8d-40e281c5c955 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:11.488086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04a0a-259f-4c9e-9a8d-40e281c5c955 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:11.489522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 686/1341] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 12 00:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:11.496210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7282-cef8-438c-9ba6-cb1c7f187dc7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:11.496535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7282-cef8-438c-9ba6-cb1c7f187dc7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:11.496783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7282-cef8-438c-9ba6-cb1c7f187dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:11.570014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-605f7282-cef8-438c-9ba6-cb1c7f187dc7 None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:11.571229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-605f7282-cef8-438c-9ba6-cb1c7f187dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:11.653639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 656/1342] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 12 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:11.662784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d04a0a-259f-4c9e-9a8d-40e281c5c955 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:11.675804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d04a0a-259f-4c9e-9a8d-40e281c5c955 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:11.679620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:11.680279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:11.680571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:11.680845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:11.689935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:11.691357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8b497-7bc8-4d8a-879c-d71d640b0cfc service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:11.700931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 687/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:15.096630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefcd70-8567-4de0-b0c6-bd469e88b4d8 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:15.097197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefcd70-8567-4de0-b0c6-bd469e88b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:15.097197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefcd70-8567-4de0-b0c6-bd469e88b4d8 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:15.098575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 657/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:15.103754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d63cf-0196-42a1-9dc6-59e42f5d0eb8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:15.104202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d63cf-0196-42a1-9dc6-59e42f5d0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:15.104397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d63cf-0196-42a1-9dc6-59e42f5d0eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:15.131605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a54d63cf-0196-42a1-9dc6-59e42f5d0eb8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:15.132807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a54d63cf-0196-42a1-9dc6-59e42f5d0eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:15.227256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 688/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:15.265018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefcd70-8567-4de0-b0c6-bd469e88b4d8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:15.277173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefcd70-8567-4de0-b0c6-bd469e88b4d8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:15.281073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:15.281694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:15.282004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:15.282251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:15.292314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:15.293788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4383-30c6-453a-a87b-4863c112ba05 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:15.303326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 658/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:19.785812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566d278-f2bf-4ae2-9531-b9ca5535e218 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:19.786389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566d278-f2bf-4ae2-9531-b9ca5535e218 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:19.786389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566d278-f2bf-4ae2-9531-b9ca5535e218 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:19.787728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 689/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:19.793184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4dcff-0e08-4918-ae1b-7330cef1c1ab None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:19.793705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4dcff-0e08-4918-ae1b-7330cef1c1ab None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:19.793705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4dcff-0e08-4918-ae1b-7330cef1c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:19.813697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-26c4dcff-0e08-4918-ae1b-7330cef1c1ab None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:19.815598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-26c4dcff-0e08-4918-ae1b-7330cef1c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:19.825055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 659/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:19.862842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566d278-f2bf-4ae2-9531-b9ca5535e218 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:19.874668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566d278-f2bf-4ae2-9531-b9ca5535e218 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:19.878549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:19.879188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:19.879468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:19.879857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:19.889082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:19.890619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230aad4-2ee6-4ee7-a0a3-14aba36fdf1f service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:19.900106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 690/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:25.209211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2adc7-a582-4219-bc69-cacda661d94e None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:25.210191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2adc7-a582-4219-bc69-cacda661d94e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:25.210671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2adc7-a582-4219-bc69-cacda661d94e None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:25.212291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 660/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:25.217675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af2e37-f5cb-402f-8d53-db369bd82fa8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:25.218089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af2e37-f5cb-402f-8d53-db369bd82fa8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:25.218219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af2e37-f5cb-402f-8d53-db369bd82fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:25.267084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-57af2e37-f5cb-402f-8d53-db369bd82fa8 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:25.268666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-57af2e37-f5cb-402f-8d53-db369bd82fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:25.362794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 691/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:29.632210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59c6f-d4fe-4906-996c-f20db33215a6 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:29.632773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59c6f-d4fe-4906-996c-f20db33215a6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:29.633141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59c6f-d4fe-4906-996c-f20db33215a6 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:29.634666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 661/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:29.640096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425efb2-c242-438a-a754-b46ba14d2262 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:29.640612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425efb2-c242-438a-a754-b46ba14d2262 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:29.641019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425efb2-c242-438a-a754-b46ba14d2262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:29.660191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7425efb2-c242-438a-a754-b46ba14d2262 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:29.661570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7425efb2-c242-438a-a754-b46ba14d2262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:29.671552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 692/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:45:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:31.194977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b353-9682-48d4-bd3c-3feec3a3a22f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:31.195533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b353-9682-48d4-bd3c-3feec3a3a22f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:31.195681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960b353-9682-48d4-bd3c-3feec3a3a22f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:31.197221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 662/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:31.202415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581bfb7-3edb-4421-ac97-4676c667d0ba None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:31.202879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581bfb7-3edb-4421-ac97-4676c667d0ba None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:31.202961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581bfb7-3edb-4421-ac97-4676c667d0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:31.222410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d581bfb7-3edb-4421-ac97-4676c667d0ba None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:31.223612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d581bfb7-3edb-4421-ac97-4676c667d0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:31.234197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 693/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:45:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:35.175840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dac58-e430-4ae3-bd44-855a2b6450ab None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:35.176439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dac58-e430-4ae3-bd44-855a2b6450ab None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:35.176439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dac58-e430-4ae3-bd44-855a2b6450ab None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:35.183495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d51d09-d788-416b-97ed-3f5776325922 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:35.184355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d51d09-d788-416b-97ed-3f5776325922 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:35.184777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d51d09-d788-416b-97ed-3f5776325922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:35.187013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 663/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:35.204461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-82d51d09-d788-416b-97ed-3f5776325922 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:35.205754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-82d51d09-d788-416b-97ed-3f5776325922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:35.213680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 694/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:35.263954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83d3f-d6da-438f-8a9e-4da3d88b9723 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:35.264303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83d3f-d6da-438f-8a9e-4da3d88b9723 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:35.264585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83d3f-d6da-438f-8a9e-4da3d88b9723 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:35.266020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 664/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 12 00:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:35.271280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3f8c-c1b1-44ab-8dba-9e2845b7f16a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:35.271561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3f8c-c1b1-44ab-8dba-9e2845b7f16a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:35.271832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b3f8c-c1b1-44ab-8dba-9e2845b7f16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:35.320655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3d0b3f8c-c1b1-44ab-8dba-9e2845b7f16a None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:35.321888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3d0b3f8c-c1b1-44ab-8dba-9e2845b7f16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:35.403523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 695/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 12 00:45:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:45:35.412157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b83d3f-d6da-438f-8a9e-4da3d88b9723 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:35.427185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b83d3f-d6da-438f-8a9e-4da3d88b9723 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:35.433894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:35.434553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:35.434844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:35.435116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:35.444613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:35.446249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd2ed2-930b-409a-a7b4-e7f22d950495 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:35.456566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 665/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:37.888714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1a34-4ef5-42f4-b8b5-410eaaaa7cde None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:37.889377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1a34-4ef5-42f4-b8b5-410eaaaa7cde None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:37.889377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1a34-4ef5-42f4-b8b5-410eaaaa7cde None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:37.891966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 696/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:37.895938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab5eb9-65ba-49b0-818b-1a0843d1978e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:37.896436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab5eb9-65ba-49b0-818b-1a0843d1978e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:37.896525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab5eb9-65ba-49b0-818b-1a0843d1978e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:37.914844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-69ab5eb9-65ba-49b0-818b-1a0843d1978e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:37.916237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-69ab5eb9-65ba-49b0-818b-1a0843d1978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:37.924689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 666/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:37.968891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac1a34-4ef5-42f4-b8b5-410eaaaa7cde None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:37.984410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac1a34-4ef5-42f4-b8b5-410eaaaa7cde None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:37.990427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:37.991163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:37.991504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:37.991836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:38.002132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:38.003790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88bfec-e7ac-44d3-95e6-511a3ba76d8a service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:38.013581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 697/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:40.532580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d17e5-ff70-4700-babc-f0180196ddb6 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:40.533156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d17e5-ff70-4700-babc-f0180196ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:40.533156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d17e5-ff70-4700-babc-f0180196ddb6 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:40.534484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 667/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:40.539861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61371-2f50-43b5-97e7-90bac645543c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:40.540343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61371-2f50-43b5-97e7-90bac645543c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:40.540454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61371-2f50-43b5-97e7-90bac645543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:40.566518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3ef61371-2f50-43b5-97e7-90bac645543c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:40.572494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3ef61371-2f50-43b5-97e7-90bac645543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:40.584449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 698/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:40.630407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804251f-715c-4a43-a0b8-2053b6f31cf0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:40.630694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804251f-715c-4a43-a0b8-2053b6f31cf0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:40.630956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804251f-715c-4a43-a0b8-2053b6f31cf0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:40.632634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 668/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 12 00:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:40.638311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f2d5-0ead-45e0-814f-1cf352b6f230 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:40.638604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f2d5-0ead-45e0-814f-1cf352b6f230 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:40.638890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f2d5-0ead-45e0-814f-1cf352b6f230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:40.659257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7d68f2d5-0ead-45e0-814f-1cf352b6f230 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:40.660513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7d68f2d5-0ead-45e0-814f-1cf352b6f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:40.671322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 699/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 12 00:45:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:40.679287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804251f-715c-4a43-a0b8-2053b6f31cf0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:40.695830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804251f-715c-4a43-a0b8-2053b6f31cf0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:40.702388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:40.703017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:40.703312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:40.703584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:40.714137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:40.715827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a5c46-e923-4672-ac8e-21d2fe998414 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:40.726302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 669/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:43.021368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cdbb9-0e3e-43f7-aa3d-2cd4dd5961f7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.021968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cdbb9-0e3e-43f7-aa3d-2cd4dd5961f7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.021968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cdbb9-0e3e-43f7-aa3d-2cd4dd5961f7 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.023551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 700/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:43.029470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c846-f837-44ad-ada9-429878820ca8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.029961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c846-f837-44ad-ada9-429878820ca8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.030053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c846-f837-44ad-ada9-429878820ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.049339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a6b1c846-f837-44ad-ada9-429878820ca8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:43.050538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a6b1c846-f837-44ad-ada9-429878820ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:43.059100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 670/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:43.104266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cdbb9-0e3e-43f7-aa3d-2cd4dd5961f7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:43.120102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cdbb9-0e3e-43f7-aa3d-2cd4dd5961f7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:43.126187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:43.126871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.127162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.127447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.135912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:43.137533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc10c1a-3a6e-45db-9322-69a7d0c20cfc service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:43.149889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 701/1371] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:43.261895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9afd-8579-4b63-9c49-4ef638adccda None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.262439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9afd-8579-4b63-9c49-4ef638adccda None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.263033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9afd-8579-4b63-9c49-4ef638adccda None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.264546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 671/1372] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:43.270800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268a34-3512-4428-bcb9-11dc239122bd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.271116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268a34-3512-4428-bcb9-11dc239122bd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.271358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268a34-3512-4428-bcb9-11dc239122bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.292439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5e268a34-3512-4428-bcb9-11dc239122bd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:43.293586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5e268a34-3512-4428-bcb9-11dc239122bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:43.304296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 702/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:43.576548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442e16-473f-41a8-b958-9fc4eed4ebb5 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.576841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442e16-473f-41a8-b958-9fc4eed4ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.577120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442e16-473f-41a8-b958-9fc4eed4ebb5 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.578902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 672/1374] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 12 00:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:43.583952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01de7-db2b-44f5-aa69-0241f0b8fff8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.584278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01de7-db2b-44f5-aa69-0241f0b8fff8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.584553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01de7-db2b-44f5-aa69-0241f0b8fff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.635166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a1d01de7-db2b-44f5-aa69-0241f0b8fff8 None None] MFA Rules not processed for user `d00e31520c5b421c8575fc063b1e6f74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:43.636557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a1d01de7-db2b-44f5-aa69-0241f0b8fff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:43.700418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 703/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 12 00:45:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:43.705992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442e16-473f-41a8-b958-9fc4eed4ebb5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:43.718333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442e16-473f-41a8-b958-9fc4eed4ebb5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:43.722045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:43.722688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:43.722966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:43.723235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:43.732243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:43.733696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74c3ed-aa00-4451-8a44-42dbf9d0031e service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:43.744015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 673/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:45.414708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9cd6-69d2-4d85-b562-197c2b4f4359 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:45.415707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9cd6-69d2-4d85-b562-197c2b4f4359 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:45.416418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9cd6-69d2-4d85-b562-197c2b4f4359 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:45.422349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 704/1377] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:45.424480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a07-43a1-4bcb-8aa1-af2ec23acdaa None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:45.425360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a07-43a1-4bcb-8aa1-af2ec23acdaa None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:45.425859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a07-43a1-4bcb-8aa1-af2ec23acdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:45.444492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-81f58a07-43a1-4bcb-8aa1-af2ec23acdaa None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:45.445916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-81f58a07-43a1-4bcb-8aa1-af2ec23acdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:45.454740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 674/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:45.500797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e9cd6-69d2-4d85-b562-197c2b4f4359 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:45.515710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e9cd6-69d2-4d85-b562-197c2b4f4359 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:45.521179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:45.521847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:45.522128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:45.522401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:45.531472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:45.533129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443f34a-378f-4aaf-b240-bb762946f474 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:45.545271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 705/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:47.820424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b4a0f-c216-4dce-a639-d85f9e85e8b0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:47.821038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b4a0f-c216-4dce-a639-d85f9e85e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:47.821038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b4a0f-c216-4dce-a639-d85f9e85e8b0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:47.822314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 675/1380] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:47.828287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47a52-f637-4b3b-b47f-57ed86760a0d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:47.829092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47a52-f637-4b3b-b47f-57ed86760a0d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:47.829339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47a52-f637-4b3b-b47f-57ed86760a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:47.847523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-12a47a52-f637-4b3b-b47f-57ed86760a0d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:47.848969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-12a47a52-f637-4b3b-b47f-57ed86760a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:47.858069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 706/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:47.902230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b4a0f-c216-4dce-a639-d85f9e85e8b0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:47.917934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b4a0f-c216-4dce-a639-d85f9e85e8b0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:47.923896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:47.924778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:47.925271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:47.925727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:47.935327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:47.937223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68210552-ec8d-447d-abad-ae053dbb841f service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:47.948665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 676/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:50.213525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cff4-d6a5-45a0-adf0-5cecf95be03c None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:50.214360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cff4-d6a5-45a0-adf0-5cecf95be03c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:50.214360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cff4-d6a5-45a0-adf0-5cecf95be03c None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:50.215494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 707/1383] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:45:50.222254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6454b-6761-4d49-8a79-8d84b5455e75 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:50.222983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6454b-6761-4d49-8a79-8d84b5455e75 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:50.222983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6454b-6761-4d49-8a79-8d84b5455e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:50.243524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-16b6454b-6761-4d49-8a79-8d84b5455e75 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:50.245914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-16b6454b-6761-4d49-8a79-8d84b5455e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:50.254409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 677/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:50.297449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376cff4-d6a5-45a0-adf0-5cecf95be03c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:50.313626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376cff4-d6a5-45a0-adf0-5cecf95be03c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:50.319949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:50.322205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:50.322604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:50.322987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:50.332524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:50.334272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1f3e-c232-42ae-a2fd-b4f5ceb9c917 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:50.344935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 708/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:52.855966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271d0f-3a1b-49a3-834b-ad2541d488e0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:52.856682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271d0f-3a1b-49a3-834b-ad2541d488e0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:52.856879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271d0f-3a1b-49a3-834b-ad2541d488e0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:52.858680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 678/1386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:45:52.866032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a48efc-5359-4e93-b475-e033dda0b550 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:52.866501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a48efc-5359-4e93-b475-e033dda0b550 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:52.866675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a48efc-5359-4e93-b475-e033dda0b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:52.891591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e7a48efc-5359-4e93-b475-e033dda0b550 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:52.893033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e7a48efc-5359-4e93-b475-e033dda0b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:52.903530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 709/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:52.962081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271d0f-3a1b-49a3-834b-ad2541d488e0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:52.977616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271d0f-3a1b-49a3-834b-ad2541d488e0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:52.983991 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:52.984702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:52.984983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:52.985255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:52.994108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:52.996788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea70b0-d248-4027-a7e3-2de86d7deab5 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:53.006517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 679/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:55.704534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3b1d-6474-4c77-8015-9ae17d08a7a9 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:55.705169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3b1d-6474-4c77-8015-9ae17d08a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:55.705169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3b1d-6474-4c77-8015-9ae17d08a7a9 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:55.706522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 710/1389] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:55.712572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27f22-97b8-4e16-930c-a964f7a85ed3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:55.713045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27f22-97b8-4e16-930c-a964f7a85ed3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:55.713193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27f22-97b8-4e16-930c-a964f7a85ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:55.733891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-23a27f22-97b8-4e16-930c-a964f7a85ed3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:55.735220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-23a27f22-97b8-4e16-930c-a964f7a85ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:55.745153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 680/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:55.791129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff3b1d-6474-4c77-8015-9ae17d08a7a9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:55.807330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff3b1d-6474-4c77-8015-9ae17d08a7a9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:55.814375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:55.815050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:55.815349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:55.815636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:55.825311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:55.826959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b28b5-e93c-4cc5-9685-7924469a17d5 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:55.837698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 711/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:58.118951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44d68-72db-4e8f-81ab-59b907730967 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:58.119859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44d68-72db-4e8f-81ab-59b907730967 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:58.119859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44d68-72db-4e8f-81ab-59b907730967 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:58.121292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 681/1392] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:58.126772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b97e2-cb94-4608-9968-515feeb9540d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:58.127233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b97e2-cb94-4608-9968-515feeb9540d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:58.127342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b97e2-cb94-4608-9968-515feeb9540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:58.146281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-af1b97e2-cb94-4608-9968-515feeb9540d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:58.147481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-af1b97e2-cb94-4608-9968-515feeb9540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:58.155500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 712/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:58.199757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44d68-72db-4e8f-81ab-59b907730967 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:58.214750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44d68-72db-4e8f-81ab-59b907730967 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:58.220631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:58.221233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:58.221525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:58.221789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:58.229944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:58.231523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc177b6-97c1-447f-a6e2-833fce79af16 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:58.240600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 682/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:45:59.412137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2c876-7dc2-4f34-968a-961d4e88bb98 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:59.412741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2c876-7dc2-4f34-968a-961d4e88bb98 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:59.412741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2c876-7dc2-4f34-968a-961d4e88bb98 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:59.414056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 713/1395] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:59.420161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0467-90b8-44bf-8736-b4a2be6ec3ee None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:59.420669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0467-90b8-44bf-8736-b4a2be6ec3ee None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:59.420770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0467-90b8-44bf-8736-b4a2be6ec3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:59.439353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8c2f0467-90b8-44bf-8736-b4a2be6ec3ee None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:59.440551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8c2f0467-90b8-44bf-8736-b4a2be6ec3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:59.448704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 683/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:59.503600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312af601-3fd1-4e1b-a2ca-4a76e6c9bd91 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:59.503965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312af601-3fd1-4e1b-a2ca-4a76e6c9bd91 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:59.504265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312af601-3fd1-4e1b-a2ca-4a76e6c9bd91 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:59.505728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 714/1397] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 12 00:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:45:59.510880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e985-0d69-4330-b3db-a856493d5cae None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:59.511167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e985-0d69-4330-b3db-a856493d5cae None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:59.511442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840e985-0d69-4330-b3db-a856493d5cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:59.557738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-4840e985-0d69-4330-b3db-a856493d5cae None None] MFA Rules not processed for user `89fc09941e0b4aa29bbfe6d41e154f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:45:59.558913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4840e985-0d69-4330-b3db-a856493d5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:45:59.617542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 684/1398] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 12 00:45:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:45:59.625587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312af601-3fd1-4e1b-a2ca-4a76e6c9bd91 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:45:59.636811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312af601-3fd1-4e1b-a2ca-4a76e6c9bd91 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:45:59.640993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:45:59.641694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:45:59.642042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:45:59.642365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:45:59.651482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:45:59.652941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8735f1b-e8d6-4058-8fc9-fb27508be781 service glance] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:45:59.662448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 715/1399] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:00.075102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c9551-d348-4360-9cbd-d65c5bc04003 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.075421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c9551-d348-4360-9cbd-d65c5bc04003 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.075691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c9551-d348-4360-9cbd-d65c5bc04003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.124857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0f9c9551-d348-4360-9cbd-d65c5bc04003 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:00.126047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0f9c9551-d348-4360-9cbd-d65c5bc04003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:00.194095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 685/1400] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:46:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:46:00.236448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fdb7-8fd9-4d1f-a047-ccd3831464fc None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.236727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fdb7-8fd9-4d1f-a047-ccd3831464fc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.237217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fdb7-8fd9-4d1f-a047-ccd3831464fc None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.238640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 716/1401] 10.222.0.57 () {54 vars in 942 bytes} [Thu May 12 00:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:00.245150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdedf1-2348-4668-a705-2a5a79e2dbfd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.245778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdedf1-2348-4668-a705-2a5a79e2dbfd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.246077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdedf1-2348-4668-a705-2a5a79e2dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.295605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-32cdedf1-2348-4668-a705-2a5a79e2dbfd None None] MFA Rules not processed for user `86ce304b33114f83a8fa63a0227aeef5`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:00.296878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-32cdedf1-2348-4668-a705-2a5a79e2dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:00.372979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 686/1402] 10.222.0.57 () {58 vars in 1039 bytes} [Thu May 12 00:46:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:00.382047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63fdb7-8fd9-4d1f-a047-ccd3831464fc None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:00.396900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63fdb7-8fd9-4d1f-a047-ccd3831464fc None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:00.402261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:00.402906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.403220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.403483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.411818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:00.413437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c0f47-e12a-474e-bb07-fa5a53588bae service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:00.422068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 717/1403] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:46:00.505305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fac6-1058-42c3-86fd-1968c7b61afa None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.505305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fac6-1058-42c3-86fd-1968c7b61afa None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.505305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fac6-1058-42c3-86fd-1968c7b61afa None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.506677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 687/1404] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:00.512515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a62c2-e89b-43a5-bd31-8797af89a3a4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.512717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a62c2-e89b-43a5-bd31-8797af89a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.513013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a62c2-e89b-43a5-bd31-8797af89a3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.537746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fb6a62c2-e89b-43a5-bd31-8797af89a3a4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:00.538977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fb6a62c2-e89b-43a5-bd31-8797af89a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:00.547306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 718/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:00.594108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fac6-1058-42c3-86fd-1968c7b61afa None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:00.610277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fac6-1058-42c3-86fd-1968c7b61afa None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:00.616581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:00.617459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:00.618107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:00.618570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:00.628981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:00.630640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac30969-72b8-4cb1-b422-b67a8fbe81b8 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:00.648457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 688/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:03.039128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9fbf-d83d-4a9b-850c-4617f2d1bcc2 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:03.039742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9fbf-d83d-4a9b-850c-4617f2d1bcc2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:03.039742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba9fbf-d83d-4a9b-850c-4617f2d1bcc2 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:03.044658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 719/1407] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:46:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:03.046643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b672ec2-2845-4fe3-b1ef-6513376aea56 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:03.047423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b672ec2-2845-4fe3-b1ef-6513376aea56 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:03.047869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b672ec2-2845-4fe3-b1ef-6513376aea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:03.066392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3b672ec2-2845-4fe3-b1ef-6513376aea56 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:03.067846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3b672ec2-2845-4fe3-b1ef-6513376aea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:03.084916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 689/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:03.125062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba9fbf-d83d-4a9b-850c-4617f2d1bcc2 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:03.140027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba9fbf-d83d-4a9b-850c-4617f2d1bcc2 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:03.149688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:03.150850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:03.151365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:03.151716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:03.161193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:03.162849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd71ef8-4329-4b69-ae2e-661f8cb7f34a service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:03.173784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 720/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:23.474760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835558-5b59-435b-a247-0a39f1067c0f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:23.475422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835558-5b59-435b-a247-0a39f1067c0f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:23.475422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835558-5b59-435b-a247-0a39f1067c0f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:23.476652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 690/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 12 00:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:23.485047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e022d5a-9982-4b07-9bc5-0008141fb217 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:23.485546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e022d5a-9982-4b07-9bc5-0008141fb217 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:23.485653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e022d5a-9982-4b07-9bc5-0008141fb217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:23.507092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1e022d5a-9982-4b07-9bc5-0008141fb217 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:23.508889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1e022d5a-9982-4b07-9bc5-0008141fb217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:23.519604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 721/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 12 00:46:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:23.539185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531abc-eb41-43a3-9a76-54ef4dddccde None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:23.539485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531abc-eb41-43a3-9a76-54ef4dddccde None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:23.539866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531abc-eb41-43a3-9a76-54ef4dddccde None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:23.542036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 691/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 12 00:46:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:23.547482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d040d8-880f-479b-b770-d0b1797a2ed7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:23.547770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d040d8-880f-479b-b770-d0b1797a2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:23.548048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d040d8-880f-479b-b770-d0b1797a2ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:23.565793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-43d040d8-880f-479b-b770-d0b1797a2ed7 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:23.566938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-43d040d8-880f-479b-b770-d0b1797a2ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:23.576758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 722/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 12 00:46:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:26.325893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12918-4eaf-4dd5-bd3e-cf6915e558f9 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:26.326475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12918-4eaf-4dd5-bd3e-cf6915e558f9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:26.326475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12918-4eaf-4dd5-bd3e-cf6915e558f9 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:26.327891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 692/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:26.333365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f3ef-a8f3-40d1-8386-3b7c7ccb06b6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:26.333929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f3ef-a8f3-40d1-8386-3b7c7ccb06b6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:26.334222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f3ef-a8f3-40d1-8386-3b7c7ccb06b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:26.353200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-76a1f3ef-a8f3-40d1-8386-3b7c7ccb06b6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:26.354387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-76a1f3ef-a8f3-40d1-8386-3b7c7ccb06b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:26.363538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 723/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:26.404994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e12918-4eaf-4dd5-bd3e-cf6915e558f9 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:26.420636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e12918-4eaf-4dd5-bd3e-cf6915e558f9 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:26.427417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:26.428129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:26.428441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:26.428724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:26.437060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:26.438641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590334cc-583e-4bec-840f-e73ed88468f9 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:26.447985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 693/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:27.431188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ede117-9172-4b08-8722-9fc3b7b01ca6 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:27.431718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ede117-9172-4b08-8722-9fc3b7b01ca6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:27.431718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ede117-9172-4b08-8722-9fc3b7b01ca6 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:27.433723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 724/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:27.442998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14950-ab0f-4bd5-b482-0df99eec557a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:27.443443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14950-ab0f-4bd5-b482-0df99eec557a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:27.443641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14950-ab0f-4bd5-b482-0df99eec557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:27.466571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8ee14950-ab0f-4bd5-b482-0df99eec557a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:27.467766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8ee14950-ab0f-4bd5-b482-0df99eec557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:27.477176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 694/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:27.651733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ede117-9172-4b08-8722-9fc3b7b01ca6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:27.669542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ede117-9172-4b08-8722-9fc3b7b01ca6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:27.676516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:27.677149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:27.677426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:27.677697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:27.686771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:27.688294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4934-65c4-4e49-b2c1-18e93fe92ff4 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:27.701776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 725/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:28.645447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac96a3-62bf-46d4-9c64-8ec23cabd389 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:28.645936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac96a3-62bf-46d4-9c64-8ec23cabd389 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:28.646059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac96a3-62bf-46d4-9c64-8ec23cabd389 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:28.647381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 695/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:28.653483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb300b-47fa-4d9e-bd6c-c77fd91772a5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:28.654153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb300b-47fa-4d9e-bd6c-c77fd91772a5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:28.654300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb300b-47fa-4d9e-bd6c-c77fd91772a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:28.673584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-60eb300b-47fa-4d9e-bd6c-c77fd91772a5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:28.674792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-60eb300b-47fa-4d9e-bd6c-c77fd91772a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:28.683665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 726/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:28.841058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac96a3-62bf-46d4-9c64-8ec23cabd389 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:28.858055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac96a3-62bf-46d4-9c64-8ec23cabd389 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:28.863963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:28.864630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:28.864915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:28.865188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:28.873790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:28.875369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03549d5a-87bf-4b2b-b4e8-a39f522c50eb service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:28.884724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 696/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:29.830413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675a9d-9e6c-4a44-832b-9672674c0fa0 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:29.830996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675a9d-9e6c-4a44-832b-9672674c0fa0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:29.830996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675a9d-9e6c-4a44-832b-9672674c0fa0 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:29.832363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 727/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:29.838604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b1fda-b1a0-40ec-a688-79cc220d0695 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:29.839339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b1fda-b1a0-40ec-a688-79cc220d0695 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:29.839489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b1fda-b1a0-40ec-a688-79cc220d0695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:29.860212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6d7b1fda-b1a0-40ec-a688-79cc220d0695 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:29.861434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6d7b1fda-b1a0-40ec-a688-79cc220d0695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:29.870053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 697/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:30.034416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675a9d-9e6c-4a44-832b-9672674c0fa0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:30.060661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675a9d-9e6c-4a44-832b-9672674c0fa0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:30.067772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:30.068441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:30.068736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:30.069005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:30.077800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:30.079420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bec4f-58db-421b-be99-801b24a73a5a service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:30.089090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 728/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:31.048573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c72ad7-75fe-4985-96cf-9351735627ae None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:31.049514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c72ad7-75fe-4985-96cf-9351735627ae None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:31.049514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c72ad7-75fe-4985-96cf-9351735627ae None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:31.051545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 698/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:31.057778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf0c9-2c8a-462e-9833-e6e71e8e4043 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:31.058538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf0c9-2c8a-462e-9833-e6e71e8e4043 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:31.058713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf0c9-2c8a-462e-9833-e6e71e8e4043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:31.084179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-f7ecf0c9-2c8a-462e-9833-e6e71e8e4043 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:31.085624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f7ecf0c9-2c8a-462e-9833-e6e71e8e4043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:31.094733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 729/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:31.251549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c72ad7-75fe-4985-96cf-9351735627ae None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:31.276049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c72ad7-75fe-4985-96cf-9351735627ae None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:31.286916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:31.287871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:31.288321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:31.288716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:31.299808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:31.301386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39107-bc6d-4407-878e-4838701bb00f service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:31.311511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 699/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:32.209113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff9d9-1e63-4e19-8f29-baee2b8a03f1 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:32.209661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff9d9-1e63-4e19-8f29-baee2b8a03f1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:32.209661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff9d9-1e63-4e19-8f29-baee2b8a03f1 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:32.211052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 730/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:32.216812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d507-0045-4c65-92a3-20aee7aa5812 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:32.217276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d507-0045-4c65-92a3-20aee7aa5812 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:32.217369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d507-0045-4c65-92a3-20aee7aa5812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:32.238094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-4411d507-0045-4c65-92a3-20aee7aa5812 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:32.239299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4411d507-0045-4c65-92a3-20aee7aa5812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:32.249943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 700/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:32.402077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff9d9-1e63-4e19-8f29-baee2b8a03f1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:32.423248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff9d9-1e63-4e19-8f29-baee2b8a03f1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:32.429698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:32.430337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:32.430621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:32.430900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:32.440189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:32.441805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7eacc7-6d03-4855-b45f-acfdae799f7f service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:32.452639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 731/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:33.348030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642513a7-ac09-4b05-9203-8a15385db69b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:33.348711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642513a7-ac09-4b05-9203-8a15385db69b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:33.348711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642513a7-ac09-4b05-9203-8a15385db69b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:33.350022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 701/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:33.356199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb91-204a-4e2f-9b6b-ccf926080c43 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:33.356707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb91-204a-4e2f-9b6b-ccf926080c43 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:33.357034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb91-204a-4e2f-9b6b-ccf926080c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:33.388729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-206ecb91-204a-4e2f-9b6b-ccf926080c43 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:33.389928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-206ecb91-204a-4e2f-9b6b-ccf926080c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:33.399226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 732/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:33.558175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642513a7-ac09-4b05-9203-8a15385db69b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:33.575475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642513a7-ac09-4b05-9203-8a15385db69b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:33.582079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:33.582713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:33.582999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:33.583274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:33.593460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:33.595045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c159db-fe70-492a-9858-f845a9f497aa service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:33.605299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 702/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:34.461972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a86d-51da-4736-954c-d3fd403078ce None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:34.462844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a86d-51da-4736-954c-d3fd403078ce None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:34.462844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a86d-51da-4736-954c-d3fd403078ce None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:34.464981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 733/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:34.470891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bcbf-a38b-4f84-8654-fa339874bec3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:34.471344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bcbf-a38b-4f84-8654-fa339874bec3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:34.471464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9bcbf-a38b-4f84-8654-fa339874bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:34.499559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f8a9bcbf-a38b-4f84-8654-fa339874bec3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:34.500801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f8a9bcbf-a38b-4f84-8654-fa339874bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:34.511168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 703/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:34.667606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a86d-51da-4736-954c-d3fd403078ce None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:34.684158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a86d-51da-4736-954c-d3fd403078ce None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:34.690868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:34.691504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:34.691784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:34.692024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:34.700378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:34.703151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745aaba-b884-4f5f-8047-76ffd775552b service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:34.720183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 734/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:35.602060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f87d17-3786-4bef-9cc8-9a94b5d22e8d None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:35.602847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f87d17-3786-4bef-9cc8-9a94b5d22e8d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:35.603288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f87d17-3786-4bef-9cc8-9a94b5d22e8d None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:35.606175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 704/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:46:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:35.612034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395263a5-5857-4feb-9f67-466bb725ba51 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:35.612590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395263a5-5857-4feb-9f67-466bb725ba51 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:35.612590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395263a5-5857-4feb-9f67-466bb725ba51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:35.631326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-395263a5-5857-4feb-9f67-466bb725ba51 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:35.632543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-395263a5-5857-4feb-9f67-466bb725ba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:35.641500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 735/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:35.788992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f87d17-3786-4bef-9cc8-9a94b5d22e8d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:35.806005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f87d17-3786-4bef-9cc8-9a94b5d22e8d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:35.813201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59eee78-4932-4967-9204-88df821a991c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:35.814078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eee78-4932-4967-9204-88df821a991c service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:35.814474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eee78-4932-4967-9204-88df821a991c service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:35.814856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eee78-4932-4967-9204-88df821a991c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:35.825770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eee78-4932-4967-9204-88df821a991c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:35.828226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eee78-4932-4967-9204-88df821a991c service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:35.841223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 705/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:37.918041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:37.918615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:37.918615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:37.920834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 736/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:37.929616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397df37-2279-4db7-b95a-3876f9a21b3d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:37.930027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397df37-2279-4db7-b95a-3876f9a21b3d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:37.930179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397df37-2279-4db7-b95a-3876f9a21b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:37.953045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-1397df37-2279-4db7-b95a-3876f9a21b3d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:37.954249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1397df37-2279-4db7-b95a-3876f9a21b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:37.963179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 706/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:38.001840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:38.004874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:38.022606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191caa5b-035e-484e-8979-a1fe871eb871 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:38.026395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:38.027032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:38.027315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:38.027591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:38.037245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:38.038671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b576c-a973-4c87-9af6-fac7540ebb62 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:38.048408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 737/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:40.461285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928e6f-41cb-4f17-bf1b-9a9f61646d4f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:40.461849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928e6f-41cb-4f17-bf1b-9a9f61646d4f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:40.461849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928e6f-41cb-4f17-bf1b-9a9f61646d4f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:40.463113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 707/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:40.469265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b3e5-0089-4cd1-be84-c82a40bc4877 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:40.469745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b3e5-0089-4cd1-be84-c82a40bc4877 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:40.469849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b3e5-0089-4cd1-be84-c82a40bc4877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:40.490255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8601b3e5-0089-4cd1-be84-c82a40bc4877 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:40.491491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8601b3e5-0089-4cd1-be84-c82a40bc4877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:40.500713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 738/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:40.540900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0928e6f-41cb-4f17-bf1b-9a9f61646d4f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:40.553166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0928e6f-41cb-4f17-bf1b-9a9f61646d4f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:40.558670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:40.559601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:40.560014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:40.560405 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:40.571003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:40.572476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84e7c0-3555-4087-b437-53261dad16de service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:40.582340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 708/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:43.125012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543bcf5-0822-4e8b-9904-eaa44bed1510 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:43.126016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543bcf5-0822-4e8b-9904-eaa44bed1510 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:43.126016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543bcf5-0822-4e8b-9904-eaa44bed1510 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:43.128825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 739/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:43.137230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54e5f3-6bb3-47b0-95f5-c6f8d6dfeed0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:43.138024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54e5f3-6bb3-47b0-95f5-c6f8d6dfeed0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:43.138211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54e5f3-6bb3-47b0-95f5-c6f8d6dfeed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:43.157459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-de54e5f3-6bb3-47b0-95f5-c6f8d6dfeed0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:43.158694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-de54e5f3-6bb3-47b0-95f5-c6f8d6dfeed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:43.167071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 709/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:43.206458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543bcf5-0822-4e8b-9904-eaa44bed1510 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:43.223685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543bcf5-0822-4e8b-9904-eaa44bed1510 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:43.230454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:43.231323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:43.231934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:43.232314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:43.244114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:43.245934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb7b0-c23b-48b4-8a15-6e744b83ae36 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:43.255810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 740/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:45.587703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a912-f7a8-4390-aecf-04782b6d05c0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:45.588299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a912-f7a8-4390-aecf-04782b6d05c0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:45.588299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604a912-f7a8-4390-aecf-04782b6d05c0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:45.589541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 710/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:45.595433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d211-d90f-4255-819b-35edd1a3ddf1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:45.595891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d211-d90f-4255-819b-35edd1a3ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:45.595997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d211-d90f-4255-819b-35edd1a3ddf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:45.614827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e472d211-d90f-4255-819b-35edd1a3ddf1 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:45.616013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e472d211-d90f-4255-819b-35edd1a3ddf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:45.625811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 741/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:45.664131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604a912-f7a8-4390-aecf-04782b6d05c0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:45.675603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604a912-f7a8-4390-aecf-04782b6d05c0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:45.679192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:45.679827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:45.680105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:45.680415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:45.689781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:45.691241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe5bd-f934-49d8-a53f-1ee76df6489c service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:45.700823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 711/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:47.947931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b96f-9310-4a7f-91bb-0465ee927322 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:47.948551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b96f-9310-4a7f-91bb-0465ee927322 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:47.948551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b96f-9310-4a7f-91bb-0465ee927322 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:47.949865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 742/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:47.957085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb03ce-3355-4b7c-a16c-3b144407bae2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:47.957544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb03ce-3355-4b7c-a16c-3b144407bae2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:47.957652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb03ce-3355-4b7c-a16c-3b144407bae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:47.979284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-dffb03ce-3355-4b7c-a16c-3b144407bae2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:47.980521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-dffb03ce-3355-4b7c-a16c-3b144407bae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:47.989350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 712/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:48.028315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b96f-9310-4a7f-91bb-0465ee927322 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:48.040362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b96f-9310-4a7f-91bb-0465ee927322 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:48.044406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:48.045035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:48.045309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:48.045591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:48.054767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:48.056199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd1618-6083-4ed0-9de6-0f1f65ced3f3 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:48.065890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 743/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:50.323882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6739d-83da-4d78-9512-43e44782b4a0 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:50.324516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6739d-83da-4d78-9512-43e44782b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:50.324516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6739d-83da-4d78-9512-43e44782b4a0 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:50.326625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 713/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:50.335489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beee54-1fe6-4fc1-bb4d-b5e523eae6d9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:50.335918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beee54-1fe6-4fc1-bb4d-b5e523eae6d9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:50.336064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beee54-1fe6-4fc1-bb4d-b5e523eae6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:50.356850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b8beee54-1fe6-4fc1-bb4d-b5e523eae6d9 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:50.358053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b8beee54-1fe6-4fc1-bb4d-b5e523eae6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:50.366289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 744/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:50.407211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6739d-83da-4d78-9512-43e44782b4a0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:50.419878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6739d-83da-4d78-9512-43e44782b4a0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:50.424231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:50.424899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:50.425192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:50.425482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:50.434522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:50.435951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d853cb-aadf-46dd-bc43-342d1dd58d49 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:50.445667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 714/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:46:52.681486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d519b-78df-4ed8-8b3a-99d37586f1ff None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:52.682378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d519b-78df-4ed8-8b3a-99d37586f1ff None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:52.682378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d519b-78df-4ed8-8b3a-99d37586f1ff None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:52.684719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 745/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:52.690347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb4c7-5a68-483b-981c-99b842c2e946 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:52.690804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb4c7-5a68-483b-981c-99b842c2e946 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:52.690914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb4c7-5a68-483b-981c-99b842c2e946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:52.709841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a3ffb4c7-5a68-483b-981c-99b842c2e946 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:52.711020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a3ffb4c7-5a68-483b-981c-99b842c2e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:52.719316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 715/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:52.757307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d519b-78df-4ed8-8b3a-99d37586f1ff None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:52.772846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d519b-78df-4ed8-8b3a-99d37586f1ff None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:52.777291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:52.778003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:52.778289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:52.778566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:52.788599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:52.790052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeee3fd-cffd-44f0-9dca-a3c6be701e14 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:52.799966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 746/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:55.050717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7400-aa6c-4b66-a637-f13ff8af5430 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:55.051276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7400-aa6c-4b66-a637-f13ff8af5430 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:55.051276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f7400-aa6c-4b66-a637-f13ff8af5430 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:55.053157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 716/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:55.058749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c53e-c149-4473-94e8-c9ef9e388e5a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:55.059215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c53e-c149-4473-94e8-c9ef9e388e5a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:55.059322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73c53e-c149-4473-94e8-c9ef9e388e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:55.077721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ca73c53e-c149-4473-94e8-c9ef9e388e5a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:55.079088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ca73c53e-c149-4473-94e8-c9ef9e388e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:55.087533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 747/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:55.127498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7400-aa6c-4b66-a637-f13ff8af5430 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:55.144143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f7400-aa6c-4b66-a637-f13ff8af5430 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:55.148747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:55.149426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:55.149705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:55.149982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:55.160462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:55.161897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54879b9-aeaf-47c8-a23b-389471eb224f service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:55.171101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 717/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:57.461289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9ed9-251e-48e8-bec2-105862877809 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:57.461892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9ed9-251e-48e8-bec2-105862877809 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:57.461892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9ed9-251e-48e8-bec2-105862877809 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:57.463791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 748/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:57.469158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0158-7062-4bf5-8e68-049f19ff958b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:57.469636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0158-7062-4bf5-8e68-049f19ff958b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:57.469741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0158-7062-4bf5-8e68-049f19ff958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:57.492951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-193e0158-7062-4bf5-8e68-049f19ff958b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:57.494139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-193e0158-7062-4bf5-8e68-049f19ff958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:57.503746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 718/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:57.551171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9ed9-251e-48e8-bec2-105862877809 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:57.563962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9ed9-251e-48e8-bec2-105862877809 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:57.568658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:57.569302 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:57.569580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:57.569849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:57.579356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:57.580836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376276f7-9cf8-433e-b634-704bab96017e service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:57.590136 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 749/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:46:59.851558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d13b-60ba-4241-8b76-973b390199a5 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:59.852512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d13b-60ba-4241-8b76-973b390199a5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:59.852512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d13b-60ba-4241-8b76-973b390199a5 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:59.854668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 719/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:46:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:46:59.860413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfa438-78e1-4df9-8a49-063fd6612248 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:59.860880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfa438-78e1-4df9-8a49-063fd6612248 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:59.860962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfa438-78e1-4df9-8a49-063fd6612248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:59.880448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-08cfa438-78e1-4df9-8a49-063fd6612248 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:46:59.881641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-08cfa438-78e1-4df9-8a49-063fd6612248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:46:59.890577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 750/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:46:59.927211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d13b-60ba-4241-8b76-973b390199a5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:46:59.939861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d13b-60ba-4241-8b76-973b390199a5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:46:59.944169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:46:59.944837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:46:59.945122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:46:59.945395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:46:59.954494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:46:59.955936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48eacf-d155-4ac5-b954-0628700d46df service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:46:59.965084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 720/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:33.218983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4a58-2913-4a0a-b54a-e4f8aa317adb None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:33.219590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4a58-2913-4a0a-b54a-e4f8aa317adb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:33.219590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4a58-2913-4a0a-b54a-e4f8aa317adb None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:33.222311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 751/1471] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:33.230336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317520f7-c07c-4327-b10f-6830f3fba121 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:33.230848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317520f7-c07c-4327-b10f-6830f3fba121 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:33.230848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317520f7-c07c-4327-b10f-6830f3fba121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:33.250946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-317520f7-c07c-4327-b10f-6830f3fba121 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:33.252155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-317520f7-c07c-4327-b10f-6830f3fba121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:33.260587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 721/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:47:33.311567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60bd92-db24-4e3f-a912-dc3a43bc59da None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:33.311838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60bd92-db24-4e3f-a912-dc3a43bc59da None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:33.312146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60bd92-db24-4e3f-a912-dc3a43bc59da None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:33.313592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 752/1473] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 12 00:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:33.319403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5963-de5e-464f-b919-c4dd009d783b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:33.319738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5963-de5e-464f-b919-c4dd009d783b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:33.320097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5963-de5e-464f-b919-c4dd009d783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:33.341184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2cfb5963-de5e-464f-b919-c4dd009d783b None None] MFA Rules not processed for user `89fc09941e0b4aa29bbfe6d41e154f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:33.342370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2cfb5963-de5e-464f-b919-c4dd009d783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:33.350121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 722/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 12 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:33.357445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60bd92-db24-4e3f-a912-dc3a43bc59da None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:33.370251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60bd92-db24-4e3f-a912-dc3a43bc59da None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:33.374844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:33.375480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:33.375767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:33.376048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:33.387356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:33.388852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b4c0c-0523-46f7-a9f6-66052273d0a0 service glance] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:33.399221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 753/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:35.684292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca608-d9d6-49d1-9cf1-af3883e6dbbe None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:35.684919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca608-d9d6-49d1-9cf1-af3883e6dbbe None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:35.684919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca608-d9d6-49d1-9cf1-af3883e6dbbe None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:35.686274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 723/1476] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:35.692159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf53-ea1c-40c8-9913-644c38c38c0a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:35.692656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf53-ea1c-40c8-9913-644c38c38c0a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:35.692769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf53-ea1c-40c8-9913-644c38c38c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:35.711943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fc3daf53-ea1c-40c8-9913-644c38c38c0a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:35.713219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fc3daf53-ea1c-40c8-9913-644c38c38c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:35.726095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 754/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:35.751986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca608-d9d6-49d1-9cf1-af3883e6dbbe None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:35.765585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca608-d9d6-49d1-9cf1-af3883e6dbbe None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:35.770766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:35.771412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:35.771694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:35.771965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:35.772739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:35.774529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:47:35.774529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:47:35.774529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:47:35.775666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ac847-0ed8-4c2f-af5d-caabeebf090a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:35.794581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 724/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 12 00:47:35 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:47:38.099030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc79288-1985-4c9c-9c39-d0d73dddd222 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:38.099583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc79288-1985-4c9c-9c39-d0d73dddd222 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:38.099583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc79288-1985-4c9c-9c39-d0d73dddd222 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:38.100928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 755/1479] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:38.106385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fff5e-fee8-4b0a-b61d-b9714dee2368 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:38.106836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fff5e-fee8-4b0a-b61d-b9714dee2368 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:38.106941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fff5e-fee8-4b0a-b61d-b9714dee2368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:38.126420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3c9fff5e-fee8-4b0a-b61d-b9714dee2368 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:38.127646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3c9fff5e-fee8-4b0a-b61d-b9714dee2368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:38.136831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 725/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:38.187551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc79288-1985-4c9c-9c39-d0d73dddd222 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:38.208690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc79288-1985-4c9c-9c39-d0d73dddd222 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:38.215394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:38.216041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:38.216347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:38.216624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:38.225504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:38.227181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec517-01ad-4d8d-bd66-40f6b3b4f355 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:38.239880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 756/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:40.685220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed20f-40c4-4255-bbe0-ee389abd9df9 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:40.686071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed20f-40c4-4255-bbe0-ee389abd9df9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:40.686239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afed20f-40c4-4255-bbe0-ee389abd9df9 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:40.687714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 726/1482] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:40.693208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dae201-70e0-4d9b-8092-c56b2727d804 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:40.693656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dae201-70e0-4d9b-8092-c56b2727d804 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:40.693776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dae201-70e0-4d9b-8092-c56b2727d804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:40.718665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-94dae201-70e0-4d9b-8092-c56b2727d804 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:40.719872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-94dae201-70e0-4d9b-8092-c56b2727d804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:40.729405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 757/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:40.768992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afed20f-40c4-4255-bbe0-ee389abd9df9 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:40.785213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afed20f-40c4-4255-bbe0-ee389abd9df9 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:40.790127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:40.790959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:40.791280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:40.791723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:40.801013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:40.802447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6dd81-21d1-4b7a-ab39-d8ef33b2a876 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:40.811720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 727/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:43.082669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b1a8-2bde-489c-85d4-2cca532ba497 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:43.083566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b1a8-2bde-489c-85d4-2cca532ba497 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:43.083566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b1a8-2bde-489c-85d4-2cca532ba497 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:43.085656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 758/1485] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:43.091057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd85b67-187a-41e6-b59f-fae2f12489b5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:43.091528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd85b67-187a-41e6-b59f-fae2f12489b5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:43.091639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd85b67-187a-41e6-b59f-fae2f12489b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:43.110291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bcd85b67-187a-41e6-b59f-fae2f12489b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:43.111467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bcd85b67-187a-41e6-b59f-fae2f12489b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:43.120411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 728/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:43.159043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b1a8-2bde-489c-85d4-2cca532ba497 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:43.170646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b1a8-2bde-489c-85d4-2cca532ba497 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:43.174693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:43.175343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:43.175631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:43.175900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:43.186859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:43.188333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e687e5e-aa64-4172-923d-8d6dd47762c5 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:43.197821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 759/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:45.609799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60598b5-89c7-4c44-9dcb-b42bfaa0410b None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:45.610690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60598b5-89c7-4c44-9dcb-b42bfaa0410b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:45.610690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60598b5-89c7-4c44-9dcb-b42bfaa0410b None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:45.612860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 729/1488] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:45.619631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702dbd-02e5-4ee8-a4a6-763a19b4c93c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:45.620458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702dbd-02e5-4ee8-a4a6-763a19b4c93c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:45.620654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702dbd-02e5-4ee8-a4a6-763a19b4c93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:45.646425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-05702dbd-02e5-4ee8-a4a6-763a19b4c93c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:45.647626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-05702dbd-02e5-4ee8-a4a6-763a19b4c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:45.656785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 760/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:45.701574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60598b5-89c7-4c44-9dcb-b42bfaa0410b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:45.714118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60598b5-89c7-4c44-9dcb-b42bfaa0410b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:45.718596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:45.719229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:45.719505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:45.719782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:45.728926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:45.730376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dc05a-e8c9-459a-9491-324285296e65 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:45.739781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 730/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:47:48.302296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc344-ad48-4f1e-a67e-eadfed21a2ee None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:48.302859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc344-ad48-4f1e-a67e-eadfed21a2ee None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:48.302859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc344-ad48-4f1e-a67e-eadfed21a2ee None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:48.305106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 761/1491] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 12 00:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:47:48.313729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57870cec-3eff-490d-a235-123cf79849dd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:48.314180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57870cec-3eff-490d-a235-123cf79849dd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:48.314318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57870cec-3eff-490d-a235-123cf79849dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:48.341764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-57870cec-3eff-490d-a235-123cf79849dd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:47:48.343432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-57870cec-3eff-490d-a235-123cf79849dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:47:48.353494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 731/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 12 00:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:47:48.395891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbc344-ad48-4f1e-a67e-eadfed21a2ee None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:47:48.411639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbc344-ad48-4f1e-a67e-eadfed21a2ee None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:47:48.416547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:47:48.417181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:47:48.417433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:47:48.417706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:47:48.432199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:47:48.433649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007bde4-fdb7-4fe5-9f91-2fe0baafbd4e service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:47:48.443622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 762/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:49:16.079473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.080105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.080105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.101952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:49:16.104805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:16.117726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 732/1494] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:49:16.128066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.128640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.128640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.147316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:49:16.148553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:16.158279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 763/1495] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:49:16.163187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.173392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43749fb-7045-44d0-ae67-db8f04b6f42a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.178538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.179261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.179546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.179853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.180623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:49:16.180918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.181942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.181942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:49:16.181942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:49:16.182831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.194849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 733/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:49:16.207042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.217115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea6b1b-b4c5-48e5-9da5-bfee5bad9005 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.223585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.224239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.224558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.224831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.225527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.227293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.227293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:49:16.227293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:49:16.228553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.272805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 764/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:49:16.277254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.287625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8269b-790a-4e4f-8015-571971108273 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.350218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.350855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.351170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.351441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.352129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.353175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.353175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:49:16.353175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:49:16.354379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.375722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 734/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:49:16.380665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.396000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d0d9e-28c8-4ef6-bdcd-4bc54ae767c4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.401816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.402480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.402762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.403034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.403758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.404817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.404817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:49:16.404817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:49:16.405690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.420500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 765/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:49:16.424850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.435048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d6d11-6795-4d5a-a9fa-5eb7094211cd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.441910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.442875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.443269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.443599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.444722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.446303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.446303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:49:16.446303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:49:16.447560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af745-e73e-4559-bac2-bb3576427f1b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.463777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 735/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:49:16.468091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.478446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ed5-5304-434a-a5a0-269a1604ce93 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.484360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.485246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.485529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.485817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] PATH_INFO: `/v3/projects/b88fcea77c67491ab839a0bfe9283a30/users/fb181763fe9d462687e5c35b2f398be3/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.502253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88fcea77c67491ab839a0bfe9283a30, user_id=fb181763fe9d462687e5c35b2f398be3, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.503225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.503225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:49:16.503225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:49:16.505181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689bddd-c510-420a-b4ee-309476ca85c0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.521380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 766/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:49:16 2022] PUT /identity/v3/projects/b88fcea77c67491ab839a0bfe9283a30/users/fb181763fe9d462687e5c35b2f398be3/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:49:16.533283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb684-c6a4-49cb-bb51-c181bc2c16da None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.533565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb684-c6a4-49cb-bb51-c181bc2c16da None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.533840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb684-c6a4-49cb-bb51-c181bc2c16da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.562737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f44eb684-c6a4-49cb-bb51-c181bc2c16da None None] MFA Rules not processed for user `fb181763fe9d462687e5c35b2f398be3`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:49:16.564035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f44eb684-c6a4-49cb-bb51-c181bc2c16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:16.646664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 736/1502] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:49:16.663973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503932f3-3898-44de-9401-41e9604c4224 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.664280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503932f3-3898-44de-9401-41e9604c4224 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.664561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503932f3-3898-44de-9401-41e9604c4224 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.665946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 767/1503] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 12 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:49:16.670962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe173a8-957d-4127-93ec-8f14ccda9b0e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.671246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe173a8-957d-4127-93ec-8f14ccda9b0e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.671535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe173a8-957d-4127-93ec-8f14ccda9b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.689964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-efe173a8-957d-4127-93ec-8f14ccda9b0e None None] MFA Rules not processed for user `d00e31520c5b421c8575fc063b1e6f74`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:49:16.691110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-efe173a8-957d-4127-93ec-8f14ccda9b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:16.746236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 737/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 12 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:49:16.754772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503932f3-3898-44de-9401-41e9604c4224 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.771137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503932f3-3898-44de-9401-41e9604c4224 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.776167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.776845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.777130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.777398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.785897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.787310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.795976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 768/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:49:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:49:16.839021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f467b6e-2169-4605-9157-e40467f4d69f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.839294 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f467b6e-2169-4605-9157-e40467f4d69f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.839569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f467b6e-2169-4605-9157-e40467f4d69f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.840943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 738/1506] 10.222.0.57 () {52 vars in 829 bytes} [Thu May 12 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:49:16.844903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:16.913497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1af1f-a074-4246-b9bf-7688813df667 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:16.918751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:16.919637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:16.919946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:16.920222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] PATH_INFO: `/v3/users/fb181763fe9d462687e5c35b2f398be3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:16.921617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb181763fe9d462687e5c35b2f398be3)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:16.922654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:16.922654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:49:16.922654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:49:16.923953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:16.994516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb181763fe9d462687e5c35b2f398be3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.013401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb181763fe9d462687e5c35b2f398be3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.014103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb181763fe9d462687e5c35b2f398be3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.014876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb181763fe9d462687e5c35b2f398be3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.033883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb181763fe9d462687e5c35b2f398be3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.054282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 769/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:49:16 2022] DELETE /identity/v3/users/fb181763fe9d462687e5c35b2f398be3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:49:17.058833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:17.059120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:17.059395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:17.076709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:49:17.077929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:17.136109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 739/1508] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:49:17.146048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:17.147727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:17.206857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c114ac7-351d-4c73-89d3-56c1a5c4d8a2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:17.211165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:17.211818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:17.212099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:17.212411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:17.222427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:17.223868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c47dc-7c02-4611-95de-a74793e077a9 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:17.233706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 770/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:49:17.611154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:49:17.612896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:49:17.632001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891da50-5c72-48b3-a663-4cf9a74eec1e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:49:17.637262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:49:17.638142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:49:17.638419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:49:17.638689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] PATH_INFO: `/v3/projects/b88fcea77c67491ab839a0bfe9283a30` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:49:17.639924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88fcea77c67491ab839a0bfe9283a30)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:49:17.640990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:49:17.640990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:49:17.640990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:49:17.642725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:49:17.703808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88fcea77c67491ab839a0bfe9283a30', 'request_id': 'req-6936b931-8bf5-4ad0-83d1-853743ff298c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.718178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88fcea77c67491ab839a0bfe9283a30', 'request_id': 'req-6936b931-8bf5-4ad0-83d1-853743ff298c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.729768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88fcea77c67491ab839a0bfe9283a30', 'request_id': 'req-6936b931-8bf5-4ad0-83d1-853743ff298c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.740539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] The token cache is being invalidate because project b88fcea77c67491ab839a0bfe9283a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:49:17.740832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.741488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6936b931-8bf5-4ad0-83d1-853743ff298c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:49:17.743501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 740/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:49:17 2022] DELETE /identity/v3/projects/b88fcea77c67491ab839a0bfe9283a30 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:50:55.409924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:55.410519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:55.410519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:55.411769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 771/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:50:55.417391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0dc96-dc94-481e-b642-633d623c6a29 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:55.417838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0dc96-dc94-481e-b642-633d623c6a29 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:55.417970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0dc96-dc94-481e-b642-633d623c6a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:55.448802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-1dc0dc96-dc94-481e-b642-633d623c6a29 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:50:55.450027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1dc0dc96-dc94-481e-b642-633d623c6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:50:55.516960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 741/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:50:55.682972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:50:55.685141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:50:55.769006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff32df-ac4d-4328-8065-b5db803c0207 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:50:55.777349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:50:55.778276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:55.778634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:55.779095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:55.788487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:50:55.790129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96f9b-616b-48d2-b611-863a7dc62008 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:50:55.800030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 772/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:50:56.723840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88add-3cc3-44fc-87fb-013d3d3fec76 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:56.724443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88add-3cc3-44fc-87fb-013d3d3fec76 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:56.724443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88add-3cc3-44fc-87fb-013d3d3fec76 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:56.725886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 742/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:50:56.731259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22f6f-4468-40fe-961a-f0f8961ddeb6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:56.731731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22f6f-4468-40fe-961a-f0f8961ddeb6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:56.731834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22f6f-4468-40fe-961a-f0f8961ddeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:56.751836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4eb22f6f-4468-40fe-961a-f0f8961ddeb6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:50:56.753077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4eb22f6f-4468-40fe-961a-f0f8961ddeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:50:56.762547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 773/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:50:56.916633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b88add-3cc3-44fc-87fb-013d3d3fec76 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:50:56.934102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b88add-3cc3-44fc-87fb-013d3d3fec76 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:50:56.941020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:50:56.941656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:56.941939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:56.942196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:56.951530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:50:56.953173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dc91f-e5e0-4e2b-a906-f4dd0d311e18 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:50:56.962990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 743/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:50:57.586882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1230c-8e03-4211-bb14-f1d1836348b7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:57.587157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1230c-8e03-4211-bb14-f1d1836348b7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:57.587422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1230c-8e03-4211-bb14-f1d1836348b7 None None] PATH_INFO: `/v3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:57.588785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 774/1517] 10.222.0.35 () {54 vars in 925 bytes} [Thu May 12 00:50:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:50:57.596788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1aeb-5522-4088-bd58-c778a62d1274 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:50:57.597101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1aeb-5522-4088-bd58-c778a62d1274 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:50:57.597377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1aeb-5522-4088-bd58-c778a62d1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:50:57.618900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ceab1aeb-5522-4088-bd58-c778a62d1274 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:50:57.620371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ceab1aeb-5522-4088-bd58-c778a62d1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:50:57.687319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 744/1518] 10.222.0.35 () {58 vars in 1015 bytes} [Thu May 12 00:50:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:01.681729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66da0-fc6a-4a5c-957f-be70394d87f7 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:01.682622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66da0-fc6a-4a5c-957f-be70394d87f7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:01.682622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66da0-fc6a-4a5c-957f-be70394d87f7 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:01.684805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 775/1519] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 12 00:51:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:01.690907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f7162-4b54-4b0b-a7ad-2bbaa2fcffab None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:01.691694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f7162-4b54-4b0b-a7ad-2bbaa2fcffab None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:01.691839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f7162-4b54-4b0b-a7ad-2bbaa2fcffab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:01.711505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d76f7162-4b54-4b0b-a7ad-2bbaa2fcffab None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:01.712751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d76f7162-4b54-4b0b-a7ad-2bbaa2fcffab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:01.722400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 745/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 12 00:51:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:01.730666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66da0-fc6a-4a5c-957f-be70394d87f7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:01.747886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66da0-fc6a-4a5c-957f-be70394d87f7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:01.754630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:01.755270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:01.755553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:01.755825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:01.764288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:01.765910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a48bff-5e94-4bc8-830b-2da1dae646d3 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:01.777668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 776/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:08.356754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59393-636d-4d1f-b1ae-c431bc507fea None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:08.357394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59393-636d-4d1f-b1ae-c431bc507fea None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:08.357394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59393-636d-4d1f-b1ae-c431bc507fea None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:08.359402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 746/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:08.367382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579b47-1642-4f85-bdad-8a4770d3caa0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:08.367822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579b47-1642-4f85-bdad-8a4770d3caa0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:08.367916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579b47-1642-4f85-bdad-8a4770d3caa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:08.389979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-48579b47-1642-4f85-bdad-8a4770d3caa0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:08.391334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-48579b47-1642-4f85-bdad-8a4770d3caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:08.402585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 777/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:08.571688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b59393-636d-4d1f-b1ae-c431bc507fea None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:08.592732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b59393-636d-4d1f-b1ae-c431bc507fea None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:08.600332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:08.601227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:08.601513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:08.601789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:08.611672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:08.613320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055132-7341-4c4c-9020-a6895443b33d service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:08.623805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 747/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:09.517988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae53bc-9e34-4735-b3a1-9b5c70ddb303 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:09.518895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae53bc-9e34-4735-b3a1-9b5c70ddb303 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:09.518895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae53bc-9e34-4735-b3a1-9b5c70ddb303 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:09.521182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 778/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 00:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:09.526593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91b30-84f5-41ac-9bd1-ed03d4446a67 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:09.527050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91b30-84f5-41ac-9bd1-ed03d4446a67 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:09.527156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a91b30-84f5-41ac-9bd1-ed03d4446a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:09.546937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-70a91b30-84f5-41ac-9bd1-ed03d4446a67 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:09.548139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-70a91b30-84f5-41ac-9bd1-ed03d4446a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:09.557221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 748/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 00:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:09.711294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae53bc-9e34-4735-b3a1-9b5c70ddb303 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:09.731175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae53bc-9e34-4735-b3a1-9b5c70ddb303 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:09.741358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:09.742227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:09.742562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:09.742891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:09.755002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:09.757381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da55a-969d-49ea-98f2-1cd0ca79ebca service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:09.767584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 779/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:13.851052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:13.851892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:13.851892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:13.854373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 749/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:51:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:13.863346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e965-e844-46ba-9207-d279a62bda9f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:13.864125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e965-e844-46ba-9207-d279a62bda9f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:13.864338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e965-e844-46ba-9207-d279a62bda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:13.886897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7424e965-e844-46ba-9207-d279a62bda9f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:13.888108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7424e965-e844-46ba-9207-d279a62bda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:13.897377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 780/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:13.945586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:13.947738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:14.007994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bc4df-7c1e-4e2a-bc87-d88c33fc0cd8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:14.011993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:14.012674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:14.012961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:14.013263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:14.022412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:14.023950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3d77a-c5bc-4534-b112-68ea8b1c0f85 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:14.033186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 750/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:16.211446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cb4-8800-4324-a5c8-b88d8aafe936 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:16.212036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cb4-8800-4324-a5c8-b88d8aafe936 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:16.212036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cb4-8800-4324-a5c8-b88d8aafe936 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:16.213390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 781/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:16.219888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa19e5-80ec-4b1c-a9f5-82451509a082 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:16.220684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa19e5-80ec-4b1c-a9f5-82451509a082 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:16.220885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa19e5-80ec-4b1c-a9f5-82451509a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:16.248792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bdfa19e5-80ec-4b1c-a9f5-82451509a082 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:16.249989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bdfa19e5-80ec-4b1c-a9f5-82451509a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:16.259709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 751/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:16.294896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e454cb4-8800-4324-a5c8-b88d8aafe936 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:16.309340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e454cb4-8800-4324-a5c8-b88d8aafe936 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:16.313918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:16.314564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:16.314863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:16.315122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:16.324215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:16.325674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d680791-cbcc-4313-94c9-a19a733a7710 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:16.335031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 782/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:20.020149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f9e2-953d-4886-898d-11036459314a None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:20.020773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f9e2-953d-4886-898d-11036459314a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:20.020773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f9e2-953d-4886-898d-11036459314a None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:20.022831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 752/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:51:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:20.029588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22a8f3-d4c3-4e68-9e65-6aa1c7a2e9be None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:20.030066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22a8f3-d4c3-4e68-9e65-6aa1c7a2e9be None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:20.030171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22a8f3-d4c3-4e68-9e65-6aa1c7a2e9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:20.050284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-bf22a8f3-d4c3-4e68-9e65-6aa1c7a2e9be None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:20.051469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bf22a8f3-d4c3-4e68-9e65-6aa1c7a2e9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:20.060187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 783/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:51:20.329656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922f9e2-953d-4886-898d-11036459314a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:20.331531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1922f9e2-953d-4886-898d-11036459314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:20.390034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922f9e2-953d-4886-898d-11036459314a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:20.394218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:20.394855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:20.395140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:20.395379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:20.404604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:20.406038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6be0c-cd97-4234-bc6c-18646168c2eb service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:20.415345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 753/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:51:22.665643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:22.666601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:22.666601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:22.669319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 784/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 00:51:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:51:22.675301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d2a4-c2dd-4bed-a8f3-275bfc732109 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:22.676024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d2a4-c2dd-4bed-a8f3-275bfc732109 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:22.676141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d2a4-c2dd-4bed-a8f3-275bfc732109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:22.699424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-56c9d2a4-c2dd-4bed-a8f3-275bfc732109 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:51:22.700678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-56c9d2a4-c2dd-4bed-a8f3-275bfc732109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:22.709401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 754/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 00:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:51:22.980308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:51:22.983654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:51:23.002814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a440249-9af6-4848-90b4-bf544bf0d474 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:51:23.007133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:51:23.007773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:51:23.008050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:51:23.008353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:51:23.018543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:51:23.020025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3794233-9271-49b7-8ce0-c9dcdd127094 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:51:23.029940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 785/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:51.728585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75588f9c-d881-4cfd-a4fa-e31c7a5079b5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.729817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75588f9c-d881-4cfd-a4fa-e31c7a5079b5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.730363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75588f9c-d881-4cfd-a4fa-e31c7a5079b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.756467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-75588f9c-d881-4cfd-a4fa-e31c7a5079b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.758113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-75588f9c-d881-4cfd-a4fa-e31c7a5079b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.773937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 755/1540] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:51.785489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1c40-5c31-4266-997f-702818a66264 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.786121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1c40-5c31-4266-997f-702818a66264 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.786121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1c40-5c31-4266-997f-702818a66264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.794686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.795369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.795871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.813428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-76fc1c40-5c31-4266-997f-702818a66264 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.815998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-76fc1c40-5c31-4266-997f-702818a66264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.825596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.827243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.834368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 786/1541] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:51.839531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64641d4-91b7-4aaf-a2bd-82afb76beadc None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.839945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64641d4-91b7-4aaf-a2bd-82afb76beadc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.840348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64641d4-91b7-4aaf-a2bd-82afb76beadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.846590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 756/1542] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:52:51.847696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:51.867453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b64641d4-91b7-4aaf-a2bd-82afb76beadc None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.873449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b64641d4-91b7-4aaf-a2bd-82afb76beadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.874737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6b3fe-2359-4579-9e62-ae582cd800d7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:51.887635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 787/1543] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:51.889140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:51.889732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60065c-8d29-4897-beb8-b21be481563a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.889853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.890098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.890371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.890756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60065c-8d29-4897-beb8-b21be481563a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.891347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60065c-8d29-4897-beb8-b21be481563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.891454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:51.891773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:51.893455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb15069-6609-4e53-be0e-74359b238693 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:51.924073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 757/1544] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:51.927202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71898-7aca-4546-8576-35352a48793d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.928353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71898-7aca-4546-8576-35352a48793d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.928638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71898-7aca-4546-8576-35352a48793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.934887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3c60065c-8d29-4897-beb8-b21be481563a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.936097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3c60065c-8d29-4897-beb8-b21be481563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.965820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 788/1545] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:51.967288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.967410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-98f71898-7aca-4546-8576-35352a48793d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.967579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.967855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.969661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-98f71898-7aca-4546-8576-35352a48793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:51.980115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 758/1546] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:51.982234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213302-8bb4-4014-8cc1-683307c6d981 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:51.982525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213302-8bb4-4014-8cc1-683307c6d981 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:51.982809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213302-8bb4-4014-8cc1-683307c6d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:51.994248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:51.996060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.005605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 789/1547] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.006755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.007925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2c213302-8bb4-4014-8cc1-683307c6d981 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.009156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2c213302-8bb4-4014-8cc1-683307c6d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.016113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0229f1-2864-4c84-89ac-5d194409a72e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.017674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 759/1548] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.019046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.019333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.019719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.021935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.022593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.022877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.023152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.023844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.025823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb16808-81ab-461d-8262-9c725e4ec1f5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.038447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.039649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.048655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 760/1549] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.049582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.052472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 790/1550] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.054706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e830d-9df2-4121-a204-e2180ca5b94b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.055200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e830d-9df2-4121-a204-e2180ca5b94b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.055328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e830d-9df2-4121-a204-e2180ca5b94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.059501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9bcca-31a1-4df5-b13d-d9a117719d9a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.073481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-987e830d-9df2-4121-a204-e2180ca5b94b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.074735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-987e830d-9df2-4121-a204-e2180ca5b94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.114866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.115546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.115833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.116110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.116876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.117183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.118837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44bd9e-1582-41fc-9ba9-63883870a714 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.137846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 761/1551] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.137846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90502-a8e9-4815-8984-cf2d330bd949 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.137846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90502-a8e9-4815-8984-cf2d330bd949 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.138229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 791/1552] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.138319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90502-a8e9-4815-8984-cf2d330bd949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.138402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20531c19-6ede-4294-8050-612f693fca04 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.138402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20531c19-6ede-4294-8050-612f693fca04 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.138402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20531c19-6ede-4294-8050-612f693fca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.157749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-20531c19-6ede-4294-8050-612f693fca04 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.158840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d3e90502-a8e9-4815-8984-cf2d330bd949 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.159064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-20531c19-6ede-4294-8050-612f693fca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.160063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d3e90502-a8e9-4815-8984-cf2d330bd949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.166918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 792/1553] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.167679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20531c19-6ede-4294-8050-612f693fca04 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.172201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 762/1554] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.172201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6e98-20bd-48a4-a00f-8202614670bd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.172201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6e98-20bd-48a4-a00f-8202614670bd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.172201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6e98-20bd-48a4-a00f-8202614670bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.177768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20531c19-6ede-4294-8050-612f693fca04 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.182142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.182795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.183076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.183346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.184068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.184400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.185423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 00:52:52.185423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:52:52.185423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:52:52.186301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.191611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-445e6e98-20bd-48a4-a00f-8202614670bd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.192838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-445e6e98-20bd-48a4-a00f-8202614670bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.198028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 793/1555] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.198875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.207632 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 763/1556] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.208087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392d6-2ecb-494d-83e7-3905c24d7f4b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.209137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.209424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.209796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.213320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.213970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.214300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.214542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.215218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.216223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:52:52.216223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:52:52.216223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:52:52.217432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96eed8-68dc-481a-a431-811797687804 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.227504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.228762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.238077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 764/1557] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.239160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.244720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 794/1558] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.248438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162e07-0dd5-4d5a-a55e-05dab940db56 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.249198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.249488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.249766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.253882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.254545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.254829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.255129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.255806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.256916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:52:52.256916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:52:52.256916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:52:52.258890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cad2b9-69d8-4158-b20d-4c0d290746a9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.267739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.268961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.277903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 795/1559] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.278675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.288907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a1c6-758a-47d2-b364-709774ba9678 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.292344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 765/1560] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.293857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.294119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.294299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.294676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.294813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.295038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.295272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.296309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.296756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.298856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.311247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 796/1561] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.319194 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.319314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:52.319314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:52.322154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5593-9bf5-4bd4-88ab-a78ab3ddf509 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.374519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 766/1562] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:52.375360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.381117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.381863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.382150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.382424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.383114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.384635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a7a91-f0a7-498c-9d99-ae2f33c92549 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.385074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.389155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.389863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.390145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.390416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.391169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.391595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.393363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.405175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 767/1563] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.405984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.415167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 797/1564] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.415167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.417899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f353-e10b-4515-9d55-38b458d1484b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.473290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.473909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25649c56-e029-45e8-aeb9-4dab820360ef admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.474041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.474217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.474488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.475210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.475509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.477169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.479058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.479713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.479991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.480274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.481035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.482637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.490000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 768/1565] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.490770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.496054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 798/1566] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:52.496054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.501839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e1cc-8e5e-40cf-9fb2-2a4b5e03efaa admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.505563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace16873-c0f1-4a73-b9b3-6ceac6b7ed01 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.507342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.508013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.508361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.508634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.509436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.509750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.510370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.511057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.511359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.511461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.511778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.512571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.514478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.522699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 769/1567] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.523505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.532978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512ede3-e803-4268-b65d-31e17118cce9 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.538477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.539131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.539414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.539687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.540414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.543685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 799/1568] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.543685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.543910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.550744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329eda-5bf9-43d6-aaf0-62f80c293b98 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.556151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.556869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.557151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.557412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.558436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:52:52.559045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.561394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.570106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 770/1569] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.570996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.584634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 800/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:52:52.584634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.596183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dea8b4-61b2-4494-b6e8-0982ab7fbf20 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.632176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859228dd-721d-4b4d-b89b-c3ff37fc93b0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.636847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.637505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.637783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.638076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.638756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.640736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.653252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.653939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.654210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.654495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.655267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.657574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.666093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 771/1571] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.666907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.683824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 801/1572] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.683824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.691664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06f803-3753-4f63-acb1-9f54a3608dc0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.728587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db371-40cd-4fb4-a5b1-04e2cc80174e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.733152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.733818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.734098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.734370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.735268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.736916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.746421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.747040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.747321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.747418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 772/1573] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:52.747593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.748107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.748294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.750201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.757830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8325e-f6b6-4e44-8bab-6a0a5b73b9fa admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.763082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.763720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.764004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.764297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.765001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.768364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.783282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 802/1574] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.784507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.799140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 773/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.800070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.845404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c209199-4262-4276-b230-b071f5de62ba admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.858014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb5fc1-3f37-4dac-8c24-2909d373e55a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.863190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.864209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.864491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.864769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.865453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.867364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.887990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 774/1576] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/users => generated 295 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.888953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.898292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892568-0e2e-4883-be58-1c26a16cb123 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.899775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.900483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.900770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.901045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.901800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.902834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.903414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.903523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.903747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.904027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.904747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.907365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.916447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 803/1577] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:52.916447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.923956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688289ff-7278-4fe9-9d0d-ad2153661879 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.928760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:52.929438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:52.929719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:52.930120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:52.930799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:52.932720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:52.935394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 775/1578] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.936209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.952590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 804/1579] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:52 2022] POST /identity/v3/users => generated 293 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:52.953369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:52.962401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccba60-3825-4301-85b3-82f497021f0f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:52.997730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c93e-b861-40cc-94f2-b0e8e2dafe05 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.002256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.002926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.003210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.003491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.004223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.005861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.015649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.016375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 776/1580] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.016607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.016888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.017159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.017264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] PATH_INFO: `/v3/projects/c1fd9534b0c84b8f85a8793caef79d3f/users/2e8eb473129a4bda9eb87010e961d4ab/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.026550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d419-11bf-4353-abc3-b30cb5554084 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.031498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fd9534b0c84b8f85a8793caef79d3f, user_id=2e8eb473129a4bda9eb87010e961d4ab, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.031638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.032330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.032683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.033293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.034180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.034472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.036761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.048212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 805/1581] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:52 2022] PUT /identity/v3/projects/c1fd9534b0c84b8f85a8793caef79d3f/users/2e8eb473129a4bda9eb87010e961d4ab/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.053176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.057421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 777/1582] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:53.058365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.069440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c67168-dc5e-4e60-8aaf-e0e91f56acb5 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.109821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254f61d-e639-40ff-a1b1-ec673fc378cb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.115055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.115723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.116000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.116289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.117003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.118934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.126070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.126753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.127034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.127314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.128044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.129679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.138354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 806/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:53.139154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.142736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 778/1584] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.142736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.149438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e43c-0e75-4a8b-94a8-418355613111 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.152510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48a4f5-8c60-4a81-aa17-24211a2a2060 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.153943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.154604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.154881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.155152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.155870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.158127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.158623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.159293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.159594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.159847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.160568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.162490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.167947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 807/1585] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.169376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.179442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60dd5-a15a-4365-be54-d68fe03593e5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.184150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.187276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 779/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:53.187276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.187485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.187485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.187485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.187485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.188440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f464-9c49-4898-ad51-4a687837e8f0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.195247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c90a5-d243-4c99-a024-f418ba6acc8e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.198891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 808/1587] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.200182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.200494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.200769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.203043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.203757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.204121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.204121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.205069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.206896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.217862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 780/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.218659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.221998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:53.223243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:53.228550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ccd8-81fb-4a0a-b38f-fbeccd55ac84 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.230846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 809/1589] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:53.231608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.233439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.234277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.235219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.235219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.236030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.238125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.242311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3926b9-048a-4609-8e27-8576106cac1a None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.246906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.247572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.247908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.248134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.251727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 781/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.251887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.252019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.252122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.259680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d7f71-2bb5-45a4-a422-53cc3ab5a85c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.263678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 810/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.263678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.264380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.265030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.265313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.265587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.266359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.267956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.271234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccbbb1-7fe4-4d1c-b216-6c393777f9b8 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.275778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.276674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.276955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.277231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] PATH_INFO: `/v3/projects/84fd26c94b524378babd79f48339f6c3/users/e2cd84fd90b74823addd853eff0322a2/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.277673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 782/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.278589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.287793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a434837-40f1-473f-af79-ee549e34d081 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.292331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.292971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.293252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.293529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.294253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.294872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84fd26c94b524378babd79f48339f6c3, user_id=e2cd84fd90b74823addd853eff0322a2, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.295812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.297663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.307520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 783/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.308410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.314906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 811/1594] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/84fd26c94b524378babd79f48339f6c3/users/e2cd84fd90b74823addd853eff0322a2/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.315741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.318769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f490aaa-9a27-4804-bd67-f473ebb563a7 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.375926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.376719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.377002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.377277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.378012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.379644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.385384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6884-0825-4bc9-bb37-e4523079e1d0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.389344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 784/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.392725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.393554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e427703-a966-4058-a701-101118c98888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.394447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427703-a966-4058-a701-101118c98888 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.394714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427703-a966-4058-a701-101118c98888 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.394983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e427703-a966-4058-a701-101118c98888 admin admin] PATH_INFO: `/v3/projects/5d280f7c68b94b91b607a4b1b750cd63/users/7facb556fc714c5fac568a21fa5ffcf8/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.410058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e427703-a966-4058-a701-101118c98888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d280f7c68b94b91b607a4b1b750cd63, user_id=7facb556fc714c5fac568a21fa5ffcf8, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.412771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e427703-a966-4058-a701-101118c98888 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.426402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 812/1596] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/5d280f7c68b94b91b607a4b1b750cd63/users/7facb556fc714c5fac568a21fa5ffcf8/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.430209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e427703-a966-4058-a701-101118c98888 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.442457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb3af-bfa4-41dc-b040-5ae84706c3ee admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.485922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e427703-a966-4058-a701-101118c98888 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.489915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.490606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.490889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.491162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.491662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.492363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.492653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.492932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.493667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.495291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.500057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.501490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.505340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 813/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.506529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.511111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 785/1598] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:53.511825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.515846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde27d2-9eb4-4b35-bacd-d5b01f158ede admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.521485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.521887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.522166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.522324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7a14a-44a0-4b26-a592-be7ebf412922 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.522434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.523150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.524770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.528295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.529207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.529491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.529771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] PATH_INFO: `/v3/projects/d50cb7a7c0d440fca827608bc2a96878/users/a71d4761b81447a194e2141442ceef1d/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.538731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 814/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.542432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.546774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50cb7a7c0d440fca827608bc2a96878, user_id=a71d4761b81447a194e2141442ceef1d, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.547934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:52:53.547934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:52:53.547934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:52:53.550046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011692fc-373c-49dd-8abf-92a1e7d1caac admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.553391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f17b4-15b7-408c-90f8-dc3dd3dd2ed7 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.558882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.559952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.560233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.560784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.561564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.564436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.564951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 786/1600] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/d50cb7a7c0d440fca827608bc2a96878/users/a71d4761b81447a194e2141442ceef1d/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.566434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dcebb-27e3-4f95-ac0d-e6ab5136555e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.566824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dcebb-27e3-4f95-ac0d-e6ab5136555e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.567198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dcebb-27e3-4f95-ac0d-e6ab5136555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.576195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 815/1601] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:53.577290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.589221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376de5f-fa77-4a4b-a578-290978fada09 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.598579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-119dcebb-27e3-4f95-ac0d-e6ab5136555e None None] MFA Rules not processed for user `e2cd84fd90b74823addd853eff0322a2`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:53.599900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-119dcebb-27e3-4f95-ac0d-e6ab5136555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:53.649569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.650477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.650857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.651024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] PATH_INFO: `/v3/projects/9d233fdd15ef41ab9e5cf40029fe5ac4/users/6a41fc2b6e1c4032bfa408eaf36887ee/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.667627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d233fdd15ef41ab9e5cf40029fe5ac4, user_id=6a41fc2b6e1c4032bfa408eaf36887ee, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.669767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 787/1602] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:53.670314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.671056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.671404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.671743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.685408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 816/1603] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/9d233fdd15ef41ab9e5cf40029fe5ac4/users/6a41fc2b6e1c4032bfa408eaf36887ee/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.686064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.690693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:53.691952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:53.756039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 788/1604] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:53.757171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b4b70-bdbf-4594-843b-93814a99f427 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.759750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.763573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.764838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.765196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.765543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] PATH_INFO: `/v3/projects/f6e1985d38344304bc3b1e8f4db5e896/users/5db47a076f5d4650b0a7a6945ae70a52/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.770039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fe266-bb5a-414b-be28-4252ce17f55e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.774666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.775584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.775860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.776144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] PATH_INFO: `/v3/projects/4ff8190444c44ef8be16282ebfb6d52e/users/31f567b22fda4bbebe63a71e23cc3c7b/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.782444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e1985d38344304bc3b1e8f4db5e896, user_id=5db47a076f5d4650b0a7a6945ae70a52, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.785154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818b1ab-3413-44ae-a3ac-9ad1d24fea92 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.793329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff8190444c44ef8be16282ebfb6d52e, user_id=31f567b22fda4bbebe63a71e23cc3c7b, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.795990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.802289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 817/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/f6e1985d38344304bc3b1e8f4db5e896/users/5db47a076f5d4650b0a7a6945ae70a52/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.802289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.802289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.802289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.817334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 789/1606] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/4ff8190444c44ef8be16282ebfb6d52e/users/31f567b22fda4bbebe63a71e23cc3c7b/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.818894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.820818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:53.822090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:53.890402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5183656-ec25-4088-bbf2-da9449b61721 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.897532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 818/1607] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:53.898559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.903318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.904383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.904738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.905015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] PATH_INFO: `/v3/projects/8cd0b83f6cb0412d80be7dfc67644b3c/users/70c00b336e284ad99da519ae4028456c/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.910408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1d3a-086e-4924-8911-d903444e9311 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:53.917149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:53.918255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.918614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.919468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.919789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.921909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.924132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd0b83f6cb0412d80be7dfc67644b3c, user_id=70c00b336e284ad99da519ae4028456c, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:53.928516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a36aca-bbe1-42af-b4b2-cacdd344d53e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:53.945856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 790/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:53 2022] PUT /identity/v3/projects/8cd0b83f6cb0412d80be7dfc67644b3c/users/70c00b336e284ad99da519ae4028456c/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:53.947251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d8841-d3a9-402d-9a8a-9d5700e2c148 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:53.947568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d8841-d3a9-402d-9a8a-9d5700e2c148 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:53.947842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d8841-d3a9-402d-9a8a-9d5700e2c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:53.956050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 819/1609] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:53.956050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:53.956050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:53.970301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-cd7d8841-d3a9-402d-9a8a-9d5700e2c148 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:53.971528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cd7d8841-d3a9-402d-9a8a-9d5700e2c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:54.022007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b606b-a767-4ae9-91c2-d72a89ce7772 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.025856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.026515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.026878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.027158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.036371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 791/1610] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:54.036371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.036371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.036371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.044053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.045545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68a171-df96-4d91-8ca2-da9024223069 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.054985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:54.056328 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:54.057403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 820/1611] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:54.058732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.059026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.059306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.064838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 792/1612] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:54.065761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.079898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1a507-4d28-4a1d-a517-414713fe394d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.085621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.086513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] MFA Rules not processed for user `31f567b22fda4bbebe63a71e23cc3c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:54.086879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.087322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.087722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:54.088061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.101969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.103496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.117204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 793/1613] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:54.118397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.128984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482bf8b-ac94-4fbb-b2e2-377af67489b8 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.134078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.134776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.135223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.135673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.136870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.138431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818172ec-b514-4676-b7ca-6da6e4706503 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.168482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 794/1614] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:54.176285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.176481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 821/1615] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:54.176587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.176587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.176779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.192324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91956261-4bea-445c-a2a4-379c73854b09 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.197674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.198626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.199664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.200764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.212150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.214238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.215474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] MFA Rules not processed for user `70c00b336e284ad99da519ae4028456c`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:54.217527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:54.226117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 822/1616] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:54.227326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.237811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6342-1d2f-44c6-8071-33b98b19b576 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.243714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.244660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.245059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.245407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.257124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.258812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.274918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 823/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:54.276867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.288498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fe09c-32bf-463b-8e41-a2e1cdebf090 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.293811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.294587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.294940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.295870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.296685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.298627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.309490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 795/1618] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:54.310125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.315859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 824/1619] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:54.316958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.324494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f79e1a-ff9b-468d-b0d6-b02c968a89b2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.330104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc9997-4589-40b8-976e-c5942848e7e6 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.332534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.333772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.334269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.334780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.335762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.336080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.336970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.337514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.338006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.338818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.339242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.341496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.377377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 796/1620] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:54.377377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.383249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 825/1621] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/projects => generated 403 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:54.384625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.465635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2999d-4934-4c9a-9e57-56963088a275 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.472112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.473181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.473568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.474510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.476369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.478400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.494449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a67cc-53ee-4d3d-97b7-97d0494ca20e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.502371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.503510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.503844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.504410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] PATH_INFO: `/v3/projects/ca9bc6f5043d4af583909950e795e96b/users/16621114489c4cf69fdf96e8397c507d/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.527943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 826/1622] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:54.529813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.543428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613a98-f3e1-4af3-897d-9b8d9e3d4761 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.548983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9bc6f5043d4af583909950e795e96b, user_id=16621114489c4cf69fdf96e8397c507d, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.550211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.550912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.551321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.551435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.552180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.552524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.566118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.600967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 797/1623] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:54 2022] PUT /identity/v3/projects/ca9bc6f5043d4af583909950e795e96b/users/16621114489c4cf69fdf96e8397c507d/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:54.602202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.617971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 827/1624] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/users => generated 308 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:52:54.627702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.658062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca3a01-7cd9-4680-b54e-04c7a2141f93 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.729518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250e371-3ac9-45c2-9fcd-d5bff803a384 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.739634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.741395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.741682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.743408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.744182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.748543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.753486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.755291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.755735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.757376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.758268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.763278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88a74-dcd8-4f0d-8b8a-bce0d865f56d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.774420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 798/1625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:54.775660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.782011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 828/1626] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:54.782011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.782011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.782011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.803068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2cdd8-4371-437c-b7de-3dd486f3d585 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.814512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] MFA Rules not processed for user `2e8eb473129a4bda9eb87010e961d4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:54.816428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:54.816556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.817223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.817516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.818321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.819593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.825359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.849199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 799/1627] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:54.850172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.860907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d0ba-08f5-420c-b7f5-27528b480b1f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.866879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:54.867895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:54.868235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:54.868659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] PATH_INFO: `/v3/projects/e57ddd5a2e5f454da7926032043a7391/users/6e56cf0577f04fb59b83fdccf55e3fe4/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:54.890118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57ddd5a2e5f454da7926032043a7391, user_id=6e56cf0577f04fb59b83fdccf55e3fe4, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:54.893340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:54.899532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 829/1628] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:54.900841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.920535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 800/1629] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:54 2022] PUT /identity/v3/projects/e57ddd5a2e5f454da7926032043a7391/users/6e56cf0577f04fb59b83fdccf55e3fe4/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:54.921429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:54.925399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633e722-0c28-4214-8673-4f958c4acbfc None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:54.995891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23312-4a67-42b0-afcb-379f12f1dade admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.001220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.002232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.002753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.003123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] PATH_INFO: `/v3/projects/1d1f7b4213784504b52d98b194027535/users/695dec1639184f87ae14b6370000f614/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.037798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1f7b4213784504b52d98b194027535, user_id=695dec1639184f87ae14b6370000f614, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.039946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.042298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2655e7a-d90a-4821-b563-9491c665964c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.043487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.044383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.045294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] PATH_INFO: `/v3/projects/ca9bc6f5043d4af583909950e795e96b/users/16621114489c4cf69fdf96e8397c507d/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.052160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9bc6f5043d4af583909950e795e96b, user_id=16621114489c4cf69fdf96e8397c507d, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.058230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.069145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 801/1630] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:54 2022] PUT /identity/v3/projects/1d1f7b4213784504b52d98b194027535/users/695dec1639184f87ae14b6370000f614/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:55.069145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.069145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.069695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.087429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 830/1631] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:54 2022] PUT /identity/v3/projects/ca9bc6f5043d4af583909950e795e96b/users/16621114489c4cf69fdf96e8397c507d/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:55.088836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.105199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] MFA Rules not processed for user `2e8eb473129a4bda9eb87010e961d4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:55.108418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:55.232431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 802/1632] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:55.233369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.251973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7d732-5628-45e7-8f24-70871b059b51 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.272547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.273264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.273543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.273818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.274566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.283248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c33b3-5ec8-4490-b5f8-347f0161890f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.311306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 831/1633] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:55.311751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4406-5ce9-47bc-8359-d9cab37f9821 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.312039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4406-5ce9-47bc-8359-d9cab37f9821 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.312339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4406-5ce9-47bc-8359-d9cab37f9821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.379418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2ebf4406-5ce9-47bc-8359-d9cab37f9821 None None] MFA Rules not processed for user `16621114489c4cf69fdf96e8397c507d`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:55.381770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2ebf4406-5ce9-47bc-8359-d9cab37f9821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:55.386565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdd1c9-b869-4356-a80c-8eb26d5f8f76 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.399541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.400841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.401190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.401543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.402355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.406109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0341b-77f3-4569-8d91-ec16920da012 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.425612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 803/1634] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:52:55.427271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbfa-ca69-4228-9887-7c3e3101ef88 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.427650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbfa-ca69-4228-9887-7c3e3101ef88 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.427968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fbfa-ca69-4228-9887-7c3e3101ef88 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.439911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 804/1635] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 12 00:52:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:55.440984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fbfa-ca69-4228-9887-7c3e3101ef88 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.455186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fbfa-ca69-4228-9887-7c3e3101ef88 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.463289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.464354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.464721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.465067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] PATH_INFO: `/v3/projects/e57ddd5a2e5f454da7926032043a7391/users/6e56cf0577f04fb59b83fdccf55e3fe4/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.467343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57ddd5a2e5f454da7926032043a7391, user_id=6e56cf0577f04fb59b83fdccf55e3fe4, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.470191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.482855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 805/1636] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:55 2022] PUT /identity/v3/projects/e57ddd5a2e5f454da7926032043a7391/users/6e56cf0577f04fb59b83fdccf55e3fe4/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:55.492610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.563219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 832/1637] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:55.564840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906df5-afca-4cdd-9c4a-5bea86b8eeeb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.565010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906df5-afca-4cdd-9c4a-5bea86b8eeeb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.565290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57906df5-afca-4cdd-9c4a-5bea86b8eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.587823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2cb3-9c12-4b21-8170-1104488cb876 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.595723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.597281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.598286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.598286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] PATH_INFO: `/v3/projects/1d1f7b4213784504b52d98b194027535/users/695dec1639184f87ae14b6370000f614/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.601045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1f7b4213784504b52d98b194027535, user_id=695dec1639184f87ae14b6370000f614, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.605027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e4ed-8e57-4b0c-8293-efd2f43a8abf admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.605854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-57906df5-afca-4cdd-9c4a-5bea86b8eeeb None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:55.610436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-57906df5-afca-4cdd-9c4a-5bea86b8eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:55.631095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 806/1638] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:52:55 2022] PUT /identity/v3/projects/1d1f7b4213784504b52d98b194027535/users/695dec1639184f87ae14b6370000f614/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:52:55.633830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.634115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.634569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.667336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] MFA Rules not processed for user `6e56cf0577f04fb59b83fdccf55e3fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:55.668681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:55.745283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 807/1639] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:55.758723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.767782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 833/1640] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 12 00:52:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:55.769371 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.769661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.769939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.833206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] MFA Rules not processed for user `695dec1639184f87ae14b6370000f614`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:55.836181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc598655-b307-433a-a944-27c34078c3a9 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.837595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:55.840133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.840834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.841114 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.841391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.912653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:55.914252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:55.931598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 808/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:55.943219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 834/1642] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:55.944187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.948150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:55.957809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e61e88-c731-407e-b1a0-04b75b56fbd2 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:55.966261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:55.967801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:55.968372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:55.968651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:55.975593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de11fd0-4fe3-4456-8411-6148bd1e4bce service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:56.002181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:56.003029 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.003161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.003422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.022982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.025348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:56.049122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.051741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:56.054227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 809/1643] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:56.055561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:56.066087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 835/1644] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:56.072693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1762bd4-2b97-4936-b2ee-ddd8689db83a service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:56.076800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:56.077537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.077930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.078572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.092662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.094687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb0e43-d050-418c-a99f-7bf3bf377fdd service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:56.108040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 810/1645] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:56.271415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:56.289670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b450e33-cbf0-4bd0-88d3-e026f07dcb49 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:56.300389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:56.301128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.301534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.302695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.462143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.463663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f471b25-1bc5-4979-99a2-c0669d6a4a0b service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:56.507379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 836/1646] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:56.598757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.599263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.599561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.602062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 811/1647] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 12 00:52:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:56.618200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.618489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.618759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.652313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:56.654523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:56.678629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 837/1648] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:52:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:52:56.693728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:56.695672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:56.780683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:56.824305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3cec-7478-42fb-89ff-8627b452e121 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:56.828230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:56.828877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.829218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.829515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.841672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.843770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21643859-80d0-4652-b543-48792a4bd867 service glance] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:56.867823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 812/1649] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:56.938131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a519-f809-4812-92e2-20cbc4970177 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:56.952317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:56.953799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:56.954714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:56.960542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] PATH_INFO: `/v3/users/70c00b336e284ad99da519ae4028456c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:56.962733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c00b336e284ad99da519ae4028456c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:56.972166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:57.085955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c00b336e284ad99da519ae4028456c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.136765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c00b336e284ad99da519ae4028456c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.139774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c00b336e284ad99da519ae4028456c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.146024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c00b336e284ad99da519ae4028456c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.168935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c00b336e284ad99da519ae4028456c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.189407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 838/1650] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:52:56 2022] DELETE /identity/v3/users/70c00b336e284ad99da519ae4028456c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:52:57.195124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.195503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.195881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.216637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:57.218217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.320106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 813/1651] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:57.349689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:57.351992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.364542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:57.365922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.382377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba658a-6658-4dfd-a260-f0134701ce17 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:57.398803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:57.399700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.399981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.400275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] PATH_INFO: `/v3/users/31f567b22fda4bbebe63a71e23cc3c7b` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.401938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f567b22fda4bbebe63a71e23cc3c7b)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:57.403798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: WARNING py.warnings [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:52:57.403798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: warnings.warn(msg) May 12 00:52:57.403798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: May 12 00:52:57.409643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:57.470909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895122f-6496-4b58-9576-237f10f44985 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:57.476779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:57.477441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.477723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.478003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.496973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:57.498797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9deafa-a978-4c11-acaa-6f51655269d2 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:57.513361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 839/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:57.515880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5476313-41ab-41a1-82a6-22000306a37d None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.516645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5476313-41ab-41a1-82a6-22000306a37d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.517149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5476313-41ab-41a1-82a6-22000306a37d None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.525207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 840/1653] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 12 00:52:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:57.527139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.527251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.528105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.545566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f567b22fda4bbebe63a71e23cc3c7b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.552029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:57.553289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.568926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 841/1654] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:52:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:57.588205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f567b22fda4bbebe63a71e23cc3c7b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.604227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:57.608987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.611329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f567b22fda4bbebe63a71e23cc3c7b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.614607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f567b22fda4bbebe63a71e23cc3c7b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.631035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4224d1b-4660-4153-9426-f3dae41f959b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f567b22fda4bbebe63a71e23cc3c7b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:52:57.637206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bace34f-8248-4c2e-ac8e-bb2a5363fb65 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:57.641038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 814/1655] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:52:57 2022] DELETE /identity/v3/users/31f567b22fda4bbebe63a71e23cc3c7b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:52:57.645881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf974ecb-b899-4531-a30b-c1153cb3237f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.646609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf974ecb-b899-4531-a30b-c1153cb3237f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.646897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf974ecb-b899-4531-a30b-c1153cb3237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.682940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bf974ecb-b899-4531-a30b-c1153cb3237f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:57.686451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bf974ecb-b899-4531-a30b-c1153cb3237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.746772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:57.747737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.747737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.748030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:57.756088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.786942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:57.788608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:57.810415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 842/1656] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:57.821306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 815/1657] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:57.963544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:57.965141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:57.986779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1ff70-b663-4a68-ae32-e546c0e68cba service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:57.993725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:57.994673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:57.995094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:57.995506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:58.016697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:58.018197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b817843-910f-404d-8184-5a4963e42cfe service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:58.039510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 843/1658] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:58.714582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684ff7-3a36-424d-af20-999afab9d8a2 None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:58.715197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684ff7-3a36-424d-af20-999afab9d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:58.715197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684ff7-3a36-424d-af20-999afab9d8a2 None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:58.716595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 816/1659] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 12 00:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:58.725781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749cf9c-8449-4be2-9633-40ec183c6bf3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:58.726098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749cf9c-8449-4be2-9633-40ec183c6bf3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:58.726375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749cf9c-8449-4be2-9633-40ec183c6bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:58.751948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8749cf9c-8449-4be2-9633-40ec183c6bf3 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:58.753610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8749cf9c-8449-4be2-9633-40ec183c6bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:58.943327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 844/1660] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 12 00:52:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:58.961847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684ff7-3a36-424d-af20-999afab9d8a2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:58.998024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684ff7-3a36-424d-af20-999afab9d8a2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:59.022515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:59.023184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.023633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.024051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.041393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:59.070772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:59.072736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bafb1-9456-4f79-b5a1-e280854c60e5 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:59.104619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 817/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:59.281202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e37a3-894a-42db-9744-c61517a1a588 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.282240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e37a3-894a-42db-9744-c61517a1a588 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.282837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e37a3-894a-42db-9744-c61517a1a588 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.285437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 845/1662] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 12 00:52:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:59.302252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9811-3721-400f-92ba-60010aaa96ee None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.302939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9811-3721-400f-92ba-60010aaa96ee None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.303447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9811-3721-400f-92ba-60010aaa96ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.343483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c13e9811-3721-400f-92ba-60010aaa96ee None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:59.347181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c13e9811-3721-400f-92ba-60010aaa96ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:59.552139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 818/1663] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 12 00:52:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:59.557924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.558486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.558927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.560826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 846/1664] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 12 00:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:52:59.565988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.566476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.566966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.602995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:52:59.605318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:59.632557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 819/1665] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:52:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:52:59.651968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:59.657800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:59.684522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e502b-c290-4cde-8bf5-3bc7376be128 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:59.694618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:59.695505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.695793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.696110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.721115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:59.722569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:52:59.749819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 847/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:52:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:52:59.911443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:52:59.913348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:52:59.961578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abc42e-1edd-4197-8e49-f386b1a84678 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:52:59.978505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:52:59.979913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:52:59.980382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:52:59.980802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] PATH_INFO: `/v3/projects/4ff8190444c44ef8be16282ebfb6d52e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:52:59.984080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff8190444c44ef8be16282ebfb6d52e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:52:59.991109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:00.107814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff8190444c44ef8be16282ebfb6d52e', 'request_id': 'req-57067050-ff17-48d5-a942-a1668601a941'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.130239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff8190444c44ef8be16282ebfb6d52e', 'request_id': 'req-57067050-ff17-48d5-a942-a1668601a941'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.155349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:00.163627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:00.174667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff8190444c44ef8be16282ebfb6d52e', 'request_id': 'req-57067050-ff17-48d5-a942-a1668601a941'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.202740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd636-abec-4f28-979d-52b53ba9db15 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:00.211173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] The token cache is being invalidate because project 4ff8190444c44ef8be16282ebfb6d52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:00.211462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.239167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.268897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 820/1667] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:52:59 2022] DELETE /identity/v3/projects/4ff8190444c44ef8be16282ebfb6d52e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:00.272241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:00.278174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:00.366672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:00.368516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:00.369124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:00.369635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] PATH_INFO: `/v3/projects/8cd0b83f6cb0412d80be7dfc67644b3c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:00.373034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd0b83f6cb0412d80be7dfc67644b3c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:00.374376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: WARNING py.warnings [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 00:53:00.374376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: warnings.warn(msg) May 12 00:53:00.374376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: May 12 00:53:00.376391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:00.378226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067050-ff17-48d5-a942-a1668601a941 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:00.383779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:00.387351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:00.387783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:00.388068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] PATH_INFO: `/v3/users/6e56cf0577f04fb59b83fdccf55e3fe4` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:00.390009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e56cf0577f04fb59b83fdccf55e3fe4)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:00.392363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:00.471089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e56cf0577f04fb59b83fdccf55e3fe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.499000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e56cf0577f04fb59b83fdccf55e3fe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.525905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e56cf0577f04fb59b83fdccf55e3fe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.526681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e56cf0577f04fb59b83fdccf55e3fe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.546568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e56cf0577f04fb59b83fdccf55e3fe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.556826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 821/1668] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:00 2022] DELETE /identity/v3/users/6e56cf0577f04fb59b83fdccf55e3fe4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:00.590217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd0b83f6cb0412d80be7dfc67644b3c', 'request_id': 'req-1da9083c-3d65-411b-b17c-f5927069b30d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.639623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd0b83f6cb0412d80be7dfc67644b3c', 'request_id': 'req-1da9083c-3d65-411b-b17c-f5927069b30d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.660970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd0b83f6cb0412d80be7dfc67644b3c', 'request_id': 'req-1da9083c-3d65-411b-b17c-f5927069b30d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.685193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] The token cache is being invalidate because project 8cd0b83f6cb0412d80be7dfc67644b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:00.685696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.686645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:00.689981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 848/1669] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:00 2022] DELETE /identity/v3/projects/8cd0b83f6cb0412d80be7dfc67644b3c => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:00.693796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:00.695882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:00.772019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:00.774447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:00.900703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ea64-0e86-48e1-b2e7-9f268a120151 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:00.934524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:00.935452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:00.942265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:00.942638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] PATH_INFO: `/v3/users/695dec1639184f87ae14b6370000f614` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:00.944627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695dec1639184f87ae14b6370000f614)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:00.946140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:01.013212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9083c-3d65-411b-b17c-f5927069b30d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:01.033878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:01.035732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:01.038545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:01.038818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] PATH_INFO: `/v3/users/e2cd84fd90b74823addd853eff0322a2` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:01.040991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2cd84fd90b74823addd853eff0322a2)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:01.043538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:01.114415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695dec1639184f87ae14b6370000f614'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.146069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695dec1639184f87ae14b6370000f614'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.160140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695dec1639184f87ae14b6370000f614'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.167736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695dec1639184f87ae14b6370000f614'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.209382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-73d82122-335f-449c-8c9f-302002e6fd47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695dec1639184f87ae14b6370000f614'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.219685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2cd84fd90b74823addd853eff0322a2'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.242989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 822/1670] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:00 2022] DELETE /identity/v3/users/695dec1639184f87ae14b6370000f614 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:01.301191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2cd84fd90b74823addd853eff0322a2'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.302170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2cd84fd90b74823addd853eff0322a2'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.302927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2cd84fd90b74823addd853eff0322a2'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.313730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2cd84fd90b74823addd853eff0322a2'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:01.399618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 849/1671] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:00 2022] DELETE /identity/v3/users/e2cd84fd90b74823addd853eff0322a2 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:01.401103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:01.401452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:01.401834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:01.443952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:01.447428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:01.563460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 823/1672] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:01.615310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:01.617705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:01.752125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6a286-3de6-473b-8a58-a7ab972f0355 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:01.758461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:01.758792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:01.759211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:01.760935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:01.788492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:01.790309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84987-8885-4c60-b0f0-2ccbe77e54d0 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:01.810565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 850/1673] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:02.332101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:02.334852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:02.377980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2612-28dc-4220-baeb-3ff77e8fddbd None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:02.388700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:02.394108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:02.395026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:02.396059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] PATH_INFO: `/v3/projects/e57ddd5a2e5f454da7926032043a7391` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:02.401920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57ddd5a2e5f454da7926032043a7391)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:02.425172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:02.660344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57ddd5a2e5f454da7926032043a7391', 'request_id': 'req-2f51b946-6e67-40f7-b488-1cf0b02dc36c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:02.688160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57ddd5a2e5f454da7926032043a7391', 'request_id': 'req-2f51b946-6e67-40f7-b488-1cf0b02dc36c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:02.711512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57ddd5a2e5f454da7926032043a7391', 'request_id': 'req-2f51b946-6e67-40f7-b488-1cf0b02dc36c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:02.758289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2e1f-85d3-4ce7-8ce6-84e1dd4261b1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:02.765648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2e1f-85d3-4ce7-8ce6-84e1dd4261b1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:02.766179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2e1f-85d3-4ce7-8ce6-84e1dd4261b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:02.769972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] The token cache is being invalidate because project e57ddd5a2e5f454da7926032043a7391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:02.770289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:02.770970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2f51b946-6e67-40f7-b488-1cf0b02dc36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:02.774053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 824/1674] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:02 2022] DELETE /identity/v3/projects/e57ddd5a2e5f454da7926032043a7391 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:02.788872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:02.789163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:02.789444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:02.834789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:02.839620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:02.852061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e24f2e1f-85d3-4ce7-8ce6-84e1dd4261b1 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:02.867674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e24f2e1f-85d3-4ce7-8ce6-84e1dd4261b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:03.018282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 851/1675] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:53:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:03.089878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 825/1676] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:03.099215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.101353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.101918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.140011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:03.150946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:03.165833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 852/1677] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:03.169841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.181176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ece164-29c5-49ba-a9b3-ee4ba5b3e8c3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.187107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.188009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.188539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.189030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.189913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:03.190358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.192144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.205315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 826/1678] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:03.209976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.226060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8aa84-4987-4dc0-b457-2b9b7dbeba72 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.231422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.232297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.232472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.232732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.234065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.238251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.282043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 853/1679] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:03 2022] POST /identity/v3/projects => generated 401 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:03.283836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.297390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900968e-c25c-409d-a605-4fc79796074e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.312878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.314683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:03.542401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.543087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.544193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.551317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.553156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.558716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.573450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe9b10-b304-4357-8f7e-06a60a38bda9 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.580473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.581878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.582728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.583112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.608442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 827/1680] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:03 2022] POST /identity/v3/users => generated 301 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:03.609597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.632024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47cf25-1375-4c12-ae9b-e7668303f0a1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.646024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.646724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.647018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.647295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.648043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.653937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.686689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 828/1681] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:03.687399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.710291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f62936-4f86-4049-800a-eb0df6b66f60 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.718830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.720300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.725501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.726257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.727237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.735270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.736958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.737455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb44f3-47a2-47e2-9abb-00230d6045a2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.758059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 854/1682] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 00:53:03.759262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:03.761931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 829/1683] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:03.783486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c651-4345-47aa-a2a6-1afa76aebe5b service swift] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:03.808327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:03.813424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.816103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.816635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] PATH_INFO: `/v3/projects/6a309165439246128288e9b2c521c757/users/bee130d891f44b00bf2ef70b6a6bbf3f/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:03.870552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a309165439246128288e9b2c521c757, user_id=bee130d891f44b00bf2ef70b6a6bbf3f, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:03.875152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:03.924717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 855/1684] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:03 2022] PUT /identity/v3/projects/6a309165439246128288e9b2c521c757/users/bee130d891f44b00bf2ef70b6a6bbf3f/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:03.937291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:03.937793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:03.938211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:04.049112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] MFA Rules not processed for user `bee130d891f44b00bf2ef70b6a6bbf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:04.052182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:04.247108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 830/1685] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:04.247670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:04.386700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8120f-a0ec-49ad-8cd5-4bdae5a7540a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:04.400512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:04.401353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:04.401710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:04.402103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:04.403310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:04.410351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:04.475206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:04.487922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 856/1686] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:04 2022] POST /identity/v3/projects => generated 399 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:04.496588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3eb5-37eb-4ba4-9bf5-c5cb9ae93225 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:04.697167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:04.697997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:04.698252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:04.698804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:04.699568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:04.710123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:04.745624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 831/1687] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:04 2022] POST /identity/v3/users => generated 306 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:04.749974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:04.763819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704f419-069c-4ef1-b63e-1f42374c87c3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:04.776582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:04.779540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:04.782094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:04.789785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:04.796177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:04.797638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:04.800803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:04.811728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:04.821291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d326f-76fd-446e-a2b7-8455d158af28 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:04.848387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:04.851281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:04.852765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:04.853212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] PATH_INFO: `/v3/projects/84fd26c94b524378babd79f48339f6c3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:04.856197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84fd26c94b524378babd79f48339f6c3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:04.862733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:04.865712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 857/1688] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:04.873299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:04.923750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a931ada-8353-4072-8069-70831f7a318e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:04.941445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:04.945166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:04.947521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:04.948014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] PATH_INFO: `/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65/users/119897b7d1fc41258f2e17f433ef5935/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.017330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b591a314bfc41fb92eab6ce05e7cd65, user_id=119897b7d1fc41258f2e17f433ef5935, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:05.023223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:05.030898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84fd26c94b524378babd79f48339f6c3', 'request_id': 'req-b5a7f729-d2ba-4468-8d62-a275bd92a79c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.064398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84fd26c94b524378babd79f48339f6c3', 'request_id': 'req-b5a7f729-d2ba-4468-8d62-a275bd92a79c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.083279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 858/1689] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:04 2022] PUT /identity/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65/users/119897b7d1fc41258f2e17f433ef5935/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:05.101319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:05.101526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84fd26c94b524378babd79f48339f6c3', 'request_id': 'req-b5a7f729-d2ba-4468-8d62-a275bd92a79c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.131204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] The token cache is being invalidate because project 84fd26c94b524378babd79f48339f6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:05.131550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.132754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.135371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 832/1690] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:04 2022] DELETE /identity/v3/projects/84fd26c94b524378babd79f48339f6c3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:05.140441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:05.142527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:05.306167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f729-d2ba-4468-8d62-a275bd92a79c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:05.346977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42d7f-e222-49d9-ac12-05f209350c11 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:05.358478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:05.359974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:05.359974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:05.360363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.361074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:05.384830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:05.403298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 859/1691] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:05.405647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:05.409249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:05.432120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9321-5653-4b2f-b28c-407cafb89cbe admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:05.438386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:05.440228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:05.440548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:05.440829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] PATH_INFO: `/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65/users/119897b7d1fc41258f2e17f433ef5935/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.443578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b591a314bfc41fb92eab6ce05e7cd65, user_id=119897b7d1fc41258f2e17f433ef5935, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:05.458974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6a92-bc72-41c2-a1d7-4391086fbdec admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:05.490713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 860/1692] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:05 2022] PUT /identity/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65/users/119897b7d1fc41258f2e17f433ef5935/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:05.491646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:05.492819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:05.492933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:05.493234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] PATH_INFO: `/v3/users/16621114489c4cf69fdf96e8397c507d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.494611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=16621114489c4cf69fdf96e8397c507d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:05.496579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:05.506053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:05.506333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:05.506627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.605409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] MFA Rules not processed for user `119897b7d1fc41258f2e17f433ef5935`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:05.606669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:05.652944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16621114489c4cf69fdf96e8397c507d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.691948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16621114489c4cf69fdf96e8397c507d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.713933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16621114489c4cf69fdf96e8397c507d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.714935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16621114489c4cf69fdf96e8397c507d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.736325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b1756b3-7635-44cf-af13-ff12d5f6e1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16621114489c4cf69fdf96e8397c507d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:05.745361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 833/1693] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:05 2022] DELETE /identity/v3/users/16621114489c4cf69fdf96e8397c507d => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:05.746850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:05.747221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:05.747422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:05.783954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:05.787223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:05.872487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 861/1694] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:05.875921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:05.879104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:05.929403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 834/1695] 10.222.0.35 () {58 vars in 1015 bytes} [Thu May 12 00:53:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:05.930331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:05.936559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.096442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b15b06-55b5-4d68-9c92-4a7f93f862e5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:06.111242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b56dd7-8b26-41f2-ba20-31c13bee41a4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:06.131492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:06.132229 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.132586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.132861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.290155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:06.291013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.291293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.291563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] PATH_INFO: `/v3/projects/1d1f7b4213784504b52d98b194027535` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.292796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1f7b4213784504b52d98b194027535)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:06.296942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:06.361741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:06.370363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:06.422622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 835/1696] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:06.423790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:06.431124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.501648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cbbb1-ab9c-4809-b503-b9aff89b8492 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:06.537403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1f7b4213784504b52d98b194027535', 'request_id': 'req-ddba6ef2-4739-4640-ae0f-0960264eb5fd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:06.553249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1f7b4213784504b52d98b194027535', 'request_id': 'req-ddba6ef2-4739-4640-ae0f-0960264eb5fd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:06.575320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1f7b4213784504b52d98b194027535', 'request_id': 'req-ddba6ef2-4739-4640-ae0f-0960264eb5fd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:06.595266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] The token cache is being invalidate because project 1d1f7b4213784504b52d98b194027535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:06.595705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:06.597539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ddba6ef2-4739-4640-ae0f-0960264eb5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:06.607294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 862/1697] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:05 2022] DELETE /identity/v3/projects/1d1f7b4213784504b52d98b194027535 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:06.609238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321121f-5c25-4e9d-bd08-cc473d206569 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.609787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321121f-5c25-4e9d-bd08-cc473d206569 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.610159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321121f-5c25-4e9d-bd08-cc473d206569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.619974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:06.621488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.621488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.621811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.626011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.646799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:06.649231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:06.666849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 836/1698] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:06.667946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:06.671896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.675529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1321121f-5c25-4e9d-bd08-cc473d206569 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:06.679040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1321121f-5c25-4e9d-bd08-cc473d206569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.785889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1ae05-9da6-4fd8-9213-88c69d55dd71 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:06.790577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:06.792480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.792791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.793248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.796805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.902345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 863/1699] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:06.904297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:06.904704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:06.905119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:06.918156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:06.921790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:06.939934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:06.944009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:06.948346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 837/1700] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:06.969412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 864/1701] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:06.973603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:06.996640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8942b-5e3c-4eea-bc58-7dd3dc60de2c service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.024611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.027358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.027653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.029856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.031269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:07.033371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.041475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.100520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 838/1702] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:07.104115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:07.115410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba26a7-693f-4b55-a193-6c3ad202ec14 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.134875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.135651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.136007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.139936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.140869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.152159 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.225465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 865/1703] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:07 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:07.227009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:07.245129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e4fec-8fc3-41fe-9222-2d04639de127 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.393346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.393979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.394266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.394945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.395241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.397506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.450773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 839/1704] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:07 2022] POST /identity/v3/users => generated 282 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:07.455100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:07.473427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb85587-baa7-4cff-87ce-2305f1a14419 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.486552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.488855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.489158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.490934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.491675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.496112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.518334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 866/1705] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:07.520335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:07.543424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98746e6a-eeab-4946-8d7b-dd85ed93d7da admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.553725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.554791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.555269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.555755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.556911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.559084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af929b-d017-44a2-a46e-72627ed053c4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.597687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 840/1706] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:07.598154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:07.625009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21059-e569-4a3a-b10e-77766df7a241 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:07.636049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:07.637396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.637853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.638349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] PATH_INFO: `/v3/projects/e24946903f73411ca0c9a46f0591b7e8/users/c4dbdfb0612c4dae856930a6ddef6b8e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.724402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24946903f73411ca0c9a46f0591b7e8, user_id=c4dbdfb0612c4dae856930a6ddef6b8e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:07.753344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:07.805960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 867/1707] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:07 2022] PUT /identity/v3/projects/e24946903f73411ca0c9a46f0591b7e8/users/c4dbdfb0612c4dae856930a6ddef6b8e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:07.818189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e118009-eb6b-4057-80b6-92bd5fe1943d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:07.818712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e118009-eb6b-4057-80b6-92bd5fe1943d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:07.818799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e118009-eb6b-4057-80b6-92bd5fe1943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:07.935370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3e118009-eb6b-4057-80b6-92bd5fe1943d None None] MFA Rules not processed for user `c4dbdfb0612c4dae856930a6ddef6b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:07.937880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3e118009-eb6b-4057-80b6-92bd5fe1943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:08.247961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:08.257588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 841/1708] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:08.815483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:08.815801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:08.816073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:08.918948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] MFA Rules not processed for user `7facb556fc714c5fac568a21fa5ffcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:08.926159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:08.938711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f49f6-66c9-4f51-9047-a0e2607aafe8 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:08.956640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:08.959446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:08.968767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:08.969400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:08.973632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:08.980146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.101891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 868/1709] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:08 2022] POST /identity/v3/projects => generated 365 bytes in 854 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:09.105770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.248354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 842/1710] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:09.278149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.283415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:09.511192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a6924-7cf1-43e7-9d1c-99d12875dffe admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:09.527307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:09.528569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:09.530116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:09.531322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:09.532355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:09.548116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.574038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3f1f-2b26-4852-b0b6-70b1e27a1c8f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:09.598289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 869/1711] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:09 2022] POST /identity/v3/users => generated 289 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:09.602778 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.634957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bae57-5494-4b7f-be8a-05d06303e9ef admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:09.641852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:09.642496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:09.643167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:09.643451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:09.644278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:09.646045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.672424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 870/1712] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:09.677322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.700580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398878c8-bc80-4e12-bef5-b78b878b84e4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:09.708465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541663-4808-4064-9591-02147824c947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:09.709961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541663-4808-4064-9591-02147824c947 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:09.710176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541663-4808-4064-9591-02147824c947 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:09.710464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541663-4808-4064-9591-02147824c947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:09.716054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:09.717008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:09.717289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:09.717561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] PATH_INFO: `/v3/projects/78ed136670774004a6a7a21648452228/users/9fce33ff24554b1d9f34db688fb77196/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:09.753182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ed136670774004a6a7a21648452228, user_id=9fce33ff24554b1d9f34db688fb77196, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:09.755936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.828851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 871/1713] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:09 2022] PUT /identity/v3/projects/78ed136670774004a6a7a21648452228/users/9fce33ff24554b1d9f34db688fb77196/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:09.834458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.853506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541663-4808-4064-9591-02147824c947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:09.857509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541663-4808-4064-9591-02147824c947 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.915703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 843/1714] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:09.920317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72316a-ce76-447c-aaea-94648c92c3e6 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:09.933451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:09.935423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:09.936487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:09.937164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:09.940087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:09.942328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a057ff9-e116-4318-897e-332c08df32c4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:09.962467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 872/1715] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:09.966595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541663-4808-4064-9591-02147824c947 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:09.982118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541663-4808-4064-9591-02147824c947 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:10.011486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:10.012779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:10.013065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:10.013340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] PATH_INFO: `/v3/projects/78ed136670774004a6a7a21648452228/users/9fce33ff24554b1d9f34db688fb77196/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:10.017656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ed136670774004a6a7a21648452228, user_id=9fce33ff24554b1d9f34db688fb77196, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:10.020890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:10.086034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:10.086286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:10.087187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:10.092063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 844/1716] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:09 2022] PUT /identity/v3/projects/78ed136670774004a6a7a21648452228/users/9fce33ff24554b1d9f34db688fb77196/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:10.162630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] MFA Rules not processed for user `9fce33ff24554b1d9f34db688fb77196`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:10.164102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:10.426308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:10.429098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 873/1717] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:10.590610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2e089-6502-4345-be03-fe10d5fbe962 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:10.605708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:10.607889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:10.609067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:10.609366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:10.632849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:10.634526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:10.683849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 845/1718] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:11.581156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:11.583659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:11.805617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c580d-05ea-4051-be6f-33142abc2d8f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:11.812645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:11.814115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:11.814590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:11.815083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] PATH_INFO: `/v3/projects/ca9bc6f5043d4af583909950e795e96b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:11.817671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9bc6f5043d4af583909950e795e96b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:11.831970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:12.019565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:12.037142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4d7e-b415-408a-9301-b01f3530d47f service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:12.042353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9bc6f5043d4af583909950e795e96b', 'request_id': 'req-2f1e91d2-7508-4da1-a2cf-bb438617d333'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.060571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9bc6f5043d4af583909950e795e96b', 'request_id': 'req-2f1e91d2-7508-4da1-a2cf-bb438617d333'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.119499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9bc6f5043d4af583909950e795e96b', 'request_id': 'req-2f1e91d2-7508-4da1-a2cf-bb438617d333'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.164559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] The token cache is being invalidate because project ca9bc6f5043d4af583909950e795e96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:12.164879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.166878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2f1e91d2-7508-4da1-a2cf-bb438617d333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.176321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 874/1719] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:11 2022] DELETE /identity/v3/projects/ca9bc6f5043d4af583909950e795e96b => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:12.182651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418d1c-e795-47e4-b4b7-4c148b167a01 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.182934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418d1c-e795-47e4-b4b7-4c148b167a01 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.183225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418d1c-e795-47e4-b4b7-4c148b167a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.248879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:12.258357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8c418d1c-e795-47e4-b4b7-4c148b167a01 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:12.259632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8c418d1c-e795-47e4-b4b7-4c148b167a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:12.260182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.260965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.261479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] PATH_INFO: `/v3/users/c4dbdfb0612c4dae856930a6ddef6b8e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.268225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4dbdfb0612c4dae856930a6ddef6b8e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:12.277897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:12.319265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 875/1720] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:12.323723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.324009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.324309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.378750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:12.380427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:12.412189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 876/1721] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:12.431387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:12.450785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4dbdfb0612c4dae856930a6ddef6b8e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.460950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb5fb3-4b7e-4a4e-8238-b7fcae886285 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:12.476092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:12.478150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.479239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.481021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.482279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:12.482582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:12.489220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:12.491884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4dbdfb0612c4dae856930a6ddef6b8e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.513227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4dbdfb0612c4dae856930a6ddef6b8e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.516871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4dbdfb0612c4dae856930a6ddef6b8e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.521812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 877/1722] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:12.528701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:12.540657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b38fc729-643d-4c3e-ad95-f3e78aba94e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4dbdfb0612c4dae856930a6ddef6b8e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:12.561916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 846/1723] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:12 2022] DELETE /identity/v3/users/c4dbdfb0612c4dae856930a6ddef6b8e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:12.564068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.564632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc3f0-6709-4241-9293-314833f18e9a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:12.565494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.565791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.621561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:12.622815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:12.772391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:12.773077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:12.773353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:12.773621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:12.775066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:12.776414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:12.839561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 847/1724] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:12.861605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 878/1725] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:12 2022] POST /identity/v3/projects => generated 387 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:12.866732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:12.868275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:12.885735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b76c8b-2dd3-4a20-988a-adf25910bbe0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:12.886963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:12.889021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:13.097748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:13.099364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.102359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.102805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.103698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:13.109136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:13.126711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473d51b-9655-4740-9766-49cee2cf691e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:13.137263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:13.137944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.156625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.157015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.172667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 848/1726] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:12 2022] POST /identity/v3/users => generated 294 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:13.178762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:13.190744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:13.198672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:13.225560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde986ee-f714-4d24-9980-1fe16fe50506 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:13.227077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 879/1727] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:13.249173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:13.253868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.254169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.255872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.258479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:13.266685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:13.319794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 849/1728] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:13.326048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:13.336018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed7c4-1f32-441d-982d-184bcaa7342e service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:13.357789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:13.358496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.358780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.359058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.359890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:13.367771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f5f26-29ef-42ef-a011-740feac152f5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:13.405366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:13.405550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 880/1729] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:13.417548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb0d8-1f7a-4cf8-b889-0b5922088766 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:13.441123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:13.442045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.442443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.442758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] PATH_INFO: `/v3/projects/5b6459c6137d444085ef9b512678ecd9/users/7d1f4ef7311e40f6842a5147d9156266/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.513599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6459c6137d444085ef9b512678ecd9, user_id=7d1f4ef7311e40f6842a5147d9156266, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:13.534450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:13.561369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 850/1730] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:13 2022] PUT /identity/v3/projects/5b6459c6137d444085ef9b512678ecd9/users/7d1f4ef7311e40f6842a5147d9156266/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:13.573238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:13.573547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:13.573812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:13.686290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] MFA Rules not processed for user `7d1f4ef7311e40f6842a5147d9156266`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:13.687581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:13.893810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 881/1731] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:13.898710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:14.118633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193edd68-1f8b-46a1-8cec-5b42ae23404c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:14.130036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:14.131533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.132153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.132940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.134189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:14.136743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ebdb7-3f4b-48a3-a95c-5feeef2bdf4b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:14.186376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 851/1732] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:13 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:14.188481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:14.262637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e6f3c-3526-447c-93bd-49dc9bfc06ae None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:14.456036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037ff0-701c-47ea-ad3b-0d57903b3d1f None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.456331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037ff0-701c-47ea-ad3b-0d57903b3d1f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.456532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037ff0-701c-47ea-ad3b-0d57903b3d1f None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.458933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 852/1733] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:53:14.529762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.529762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.529762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.553741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:14.554596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.554997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.555386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.556241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:14.558730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:14.616176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:14.620052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:14.691902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 882/1734] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:14 2022] POST /identity/v3/users => generated 298 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:14.697921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:14.755512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce080a36-ffd5-4450-93bf-dca6a9b2429c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:14.791318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:14.792026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.793334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.793688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.794517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:14.796280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:14.811713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 853/1735] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:53:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:14.853679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 883/1736] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:14.859115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:14.873538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3f173-28bd-425a-86c3-c693504080fb None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:14.880775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:14.882299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:14.882579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:14.882868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] PATH_INFO: `/v3/projects/9eaedb848aec4e2d81bc106e894acbd5/users/550805d9103f4aa59cc9589aaeb2e777/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:14.887928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:14.892737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:14.914210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eaedb848aec4e2d81bc106e894acbd5, user_id=550805d9103f4aa59cc9589aaeb2e777, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:14.917072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:14.935092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b52ea-2e5c-4937-bca1-5649d6dd3abb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:14.958994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 854/1737] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:14 2022] PUT /identity/v3/projects/9eaedb848aec4e2d81bc106e894acbd5/users/550805d9103f4aa59cc9589aaeb2e777/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:14.960132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:15.119593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:15.120653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.121011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.121380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] PATH_INFO: `/v3/users/bee130d891f44b00bf2ef70b6a6bbf3f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.124495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee130d891f44b00bf2ef70b6a6bbf3f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:15.127289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:15.216005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76df2-d83d-4322-99a2-b234bdd9ab35 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:15.231977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:15.232894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.233327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.248837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.249656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:15.254195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:15.282692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 855/1738] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:15.283275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee130d891f44b00bf2ef70b6a6bbf3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:15.283631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:15.287354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:15.321912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee130d891f44b00bf2ef70b6a6bbf3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:15.324600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee130d891f44b00bf2ef70b6a6bbf3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:15.338768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee130d891f44b00bf2ef70b6a6bbf3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:15.381471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee130d891f44b00bf2ef70b6a6bbf3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:15.430937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 884/1739] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:14 2022] DELETE /identity/v3/users/bee130d891f44b00bf2ef70b6a6bbf3f => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:15.433968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:15.440701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:15.643806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3e30c-dd0d-4b56-94ee-2de46e8a6b16 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:15.649626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff899fc5-53d9-4573-b0cd-b01f7b09289d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:15.660681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:15.662147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.662588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.662870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] PATH_INFO: `/v3/projects/9eaedb848aec4e2d81bc106e894acbd5/users/550805d9103f4aa59cc9589aaeb2e777/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.668176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eaedb848aec4e2d81bc106e894acbd5, user_id=550805d9103f4aa59cc9589aaeb2e777, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:15.674517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058043b-88cc-49e6-82ed-291bb4bdfe0a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:15.710783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 885/1740] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:15 2022] PUT /identity/v3/projects/9eaedb848aec4e2d81bc106e894acbd5/users/550805d9103f4aa59cc9589aaeb2e777/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:15.713375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2797bd-d95d-4900-b265-ebfce5955278 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.713669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2797bd-d95d-4900-b265-ebfce5955278 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.713947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2797bd-d95d-4900-b265-ebfce5955278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.794453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0f2797bd-d95d-4900-b265-ebfce5955278 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:15.797929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f2797bd-d95d-4900-b265-ebfce5955278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:15.842810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:15.843873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.845323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.845825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.848590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:15.871498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:15.876770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49931e-803b-4020-9781-f80aa417c4e9 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:15.899258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 856/1741] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:15.901129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:15.901672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:15.901909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:15.994577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] MFA Rules not processed for user `a71d4761b81447a194e2141442ceef1d`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:15.995885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.158391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 886/1742] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:16.159653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:16.159907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:16.160185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:16.196441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 857/1743] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:16.207294 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:16.218058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.254803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cf3f2-6228-4c37-9eec-3b6df6e187f2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:16.262904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:16.263858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:16.264192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:16.264505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] PATH_INFO: `/v3/projects/e24946903f73411ca0c9a46f0591b7e8` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:16.266370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24946903f73411ca0c9a46f0591b7e8)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:16.270396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:16.330015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] MFA Rules not processed for user `550805d9103f4aa59cc9589aaeb2e777`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:16.331293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.358760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24946903f73411ca0c9a46f0591b7e8', 'request_id': 'req-dff91240-6b25-4bcc-b823-fc6740693e35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:16.378446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24946903f73411ca0c9a46f0591b7e8', 'request_id': 'req-dff91240-6b25-4bcc-b823-fc6740693e35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:16.406574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24946903f73411ca0c9a46f0591b7e8', 'request_id': 'req-dff91240-6b25-4bcc-b823-fc6740693e35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:16.464471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] The token cache is being invalidate because project e24946903f73411ca0c9a46f0591b7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:16.466325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:16.468754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dff91240-6b25-4bcc-b823-fc6740693e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:16.476113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 858/1744] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:16 2022] DELETE /identity/v3/projects/e24946903f73411ca0c9a46f0591b7e8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:16.481845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:16.482707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:16.484622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:16.601900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] MFA Rules not processed for user `6a41fc2b6e1c4032bfa408eaf36887ee`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:16.605598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.651862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 887/1745] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:16.655650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:16.659656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.829611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206b54c-f7a8-4132-ad06-5b67ee189cd0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:16.838634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:16.839316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:16.839594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:16.839867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:16.846519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 859/1746] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:16.847230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:16.853179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:16.856615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.009667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:17.011197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:17.056696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 888/1747] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:17.057676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:17.079893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.120188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8d92f-145b-49ca-9a22-86e6e06cd140 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:17.120614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733e8e-a763-4684-8b6a-a28ff3955346 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:17.135149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:17.136343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:17.136899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:17.137318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] PATH_INFO: `/v3/users/9fce33ff24554b1d9f34db688fb77196` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:17.141337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fce33ff24554b1d9f34db688fb77196)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:17.150016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:17.150713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:17.150997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:17.151748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:17.152608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:17.158735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.348351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:17.351837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:17.390603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 860/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:17.392639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:17.398696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fce33ff24554b1d9f34db688fb77196'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:17.400467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.428425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fce33ff24554b1d9f34db688fb77196'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:17.429584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fce33ff24554b1d9f34db688fb77196'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:17.434246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fce33ff24554b1d9f34db688fb77196'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:17.448637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fce33ff24554b1d9f34db688fb77196'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:17.477989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 889/1749] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:17 2022] DELETE /identity/v3/users/9fce33ff24554b1d9f34db688fb77196 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:17.480174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:17.483771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.585907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc522d90-7771-4c66-9f46-cb7f0bb36c84 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:17.702437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:17.703385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:17.703783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:17.704177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:17.717351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.751814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692215a-6878-4a9d-8787-dddcb212f809 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:17.759895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:17.762087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:17.762781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:17.763258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:17.780537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:17.863993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:17.866071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:17.887445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 861/1750] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:17.926873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:17.929111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568015e8-f26a-4518-bb8f-48cd004d4643 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:17.953262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 890/1751] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:17.958575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:17.961130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:18.121350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50efca-3948-42fb-aed1-9fa13b6d6356 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:18.127996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:18.130275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:18.130798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:18.132197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] PATH_INFO: `/v3/users/7d1f4ef7311e40f6842a5147d9156266` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:18.157036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1f4ef7311e40f6842a5147d9156266)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:18.160046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:18.323677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1f4ef7311e40f6842a5147d9156266'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:18.364162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1f4ef7311e40f6842a5147d9156266'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:18.419496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1f4ef7311e40f6842a5147d9156266'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:18.422089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1f4ef7311e40f6842a5147d9156266'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:18.458069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1f4ef7311e40f6842a5147d9156266'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:18.506680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 862/1752] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:17 2022] DELETE /identity/v3/users/7d1f4ef7311e40f6842a5147d9156266 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:18.511097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:18.511817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:18.511817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:18.557889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:18.563099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:18.774962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 891/1753] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:18.816729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:18.818805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:18.975166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf2c45-251d-4662-875a-566e25dfa9d4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:18.990926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:18.991728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:18.992019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:18.992324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:19.029259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:19.030807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:19.061738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 863/1754] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:20.116526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:20.118935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:20.150076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb71cb-a58d-4b83-ad26-af47f0d8fdee None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:20.167161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:20.169272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:20.169821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:20.175697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] PATH_INFO: `/v3/projects/6a309165439246128288e9b2c521c757` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:20.177668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a309165439246128288e9b2c521c757)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:20.180067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:20.281261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a309165439246128288e9b2c521c757', 'request_id': 'req-75c70510-b67e-4d69-bad8-caa0dfceb714'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.310428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a309165439246128288e9b2c521c757', 'request_id': 'req-75c70510-b67e-4d69-bad8-caa0dfceb714'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.335764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a309165439246128288e9b2c521c757', 'request_id': 'req-75c70510-b67e-4d69-bad8-caa0dfceb714'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.371175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] The token cache is being invalidate because project 6a309165439246128288e9b2c521c757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:20.371467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.373090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-75c70510-b67e-4d69-bad8-caa0dfceb714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.388732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:20.400171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 892/1755] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:20 2022] DELETE /identity/v3/projects/6a309165439246128288e9b2c521c757 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:20.402900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa30f-e6d0-496e-981d-95085ea7eff3 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:20.492467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:20.492467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:20.492713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:20.493044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] PATH_INFO: `/v3/users/119897b7d1fc41258f2e17f433ef5935` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:20.494541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=119897b7d1fc41258f2e17f433ef5935)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:20.508969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:20.596050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119897b7d1fc41258f2e17f433ef5935'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.611902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119897b7d1fc41258f2e17f433ef5935'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.621263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119897b7d1fc41258f2e17f433ef5935'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.622550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119897b7d1fc41258f2e17f433ef5935'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.643519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119897b7d1fc41258f2e17f433ef5935'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:20.664738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 864/1756] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:20 2022] DELETE /identity/v3/users/119897b7d1fc41258f2e17f433ef5935 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:20.769252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:20.769585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:20.770053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:20.802102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] MFA Rules not processed for user `7facb556fc714c5fac568a21fa5ffcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:20.805339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:20.930167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 893/1757] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:20.974860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:20.977162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:21.201433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7711e1-f9ed-43de-9a2f-cf9e9b2e3400 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:21.221376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:21.222295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.222573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.222843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] PATH_INFO: `/v3/projects/5b6459c6137d444085ef9b512678ecd9` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.225093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6459c6137d444085ef9b512678ecd9)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:21.227060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:21.236866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:21.239197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:21.269604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35decb08-3139-4f57-aeac-7ffb42f0d5e0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:21.283419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:21.284704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.285164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.285630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] PATH_INFO: `/v3/projects/78ed136670774004a6a7a21648452228` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.290874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ed136670774004a6a7a21648452228)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:21.296243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:21.397073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6459c6137d444085ef9b512678ecd9', 'request_id': 'req-261ae487-95ca-40a1-a18e-260cef35e954'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.405324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6459c6137d444085ef9b512678ecd9', 'request_id': 'req-261ae487-95ca-40a1-a18e-260cef35e954'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.422463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6459c6137d444085ef9b512678ecd9', 'request_id': 'req-261ae487-95ca-40a1-a18e-260cef35e954'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.452813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] The token cache is being invalidate because project 5b6459c6137d444085ef9b512678ecd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:21.453133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.457966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.462863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 865/1758] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:20 2022] DELETE /identity/v3/projects/5b6459c6137d444085ef9b512678ecd9 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:21.466582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:21.470168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:21.473014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ed136670774004a6a7a21648452228', 'request_id': 'req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.502395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ed136670774004a6a7a21648452228', 'request_id': 'req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.540152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ed136670774004a6a7a21648452228', 'request_id': 'req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.577933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] The token cache is being invalidate because project 78ed136670774004a6a7a21648452228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:21.578152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.578868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5629715c-0fb3-4f2c-a8d8-1c0754aa36a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.584268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 894/1759] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:21 2022] DELETE /identity/v3/projects/78ed136670774004a6a7a21648452228 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:21.590688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cefd49-4962-4801-8916-c17043f3410d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.590959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cefd49-4962-4801-8916-c17043f3410d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.593270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cefd49-4962-4801-8916-c17043f3410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.648830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d0cefd49-4962-4801-8916-c17043f3410d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:21.649838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ae487-95ca-40a1-a18e-260cef35e954 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:21.650218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d0cefd49-4962-4801-8916-c17043f3410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:21.744324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:21.746492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.746932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.747642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] PATH_INFO: `/v3/users/550805d9103f4aa59cc9589aaeb2e777` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.749331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=550805d9103f4aa59cc9589aaeb2e777)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:21.753238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:21.768139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 895/1760] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:21.778020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.778020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.778020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.799461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:21.800824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:21.818074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 896/1761] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:21.819505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:21.831062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e615e-44a3-4a63-82a3-02c904d7d37c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:21.837828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:21.838974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.839195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.839473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.840752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:21.840752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:21.842321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:21.846753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550805d9103f4aa59cc9589aaeb2e777'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.864101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 897/1762] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:21.865287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:21.879076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550805d9103f4aa59cc9589aaeb2e777'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.882198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688227ed-153b-491f-8869-31a4075b9cf9 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:21.887222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:21.887900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:21.888177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:21.888494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:21.889562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550805d9103f4aa59cc9589aaeb2e777'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.890222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:21.890710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550805d9103f4aa59cc9589aaeb2e777'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.892912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:21.909058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550805d9103f4aa59cc9589aaeb2e777'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:21.945780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 898/1764] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:21 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:21.946004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 866/1764] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:21 2022] DELETE /identity/v3/users/550805d9103f4aa59cc9589aaeb2e777 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:21.953249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:21.955192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.038800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194ee88-1d72-4853-9e76-e2a781adacf1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:22.046024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:22.046921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.047465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.047759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.048529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:22.051171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf495e-f8e4-40b8-891f-e5efcde8ba38 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:22.091244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 899/1765] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:21 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:22.095379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:22.184868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae306f-f3f5-44ec-87e1-2703425ee7ce admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:22.188161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.188766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.189109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.211620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:22.212697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.213716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.214196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.215832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:22.216843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:22.218028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.221070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51ac76-6896-4e7b-b764-58b63fcacf0c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:22.238517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 867/1766] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:22.240202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d40ac-da5c-4868-9d4f-440b1449fe6b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.240816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d40ac-da5c-4868-9d4f-440b1449fe6b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.241475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d40ac-da5c-4868-9d4f-440b1449fe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.288470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-e17d40ac-da5c-4868-9d4f-440b1449fe6b None None] MFA Rules not processed for user `5db47a076f5d4650b0a7a6945ae70a52`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:22.290142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e17d40ac-da5c-4868-9d4f-440b1449fe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.358804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 900/1767] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:53:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:22.367543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:22.406246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9d216-b954-4c94-8619-8122a85c0768 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:22.422349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:22.423142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.423523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.423898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.434525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:22.434525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:22.436147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 868/1768] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:22.439411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99084183-176f-41c3-b5a7-1b7583b308c0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.443569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99084183-176f-41c3-b5a7-1b7583b308c0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.444076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99084183-176f-41c3-b5a7-1b7583b308c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.447412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 901/1769] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:22.451629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:22.482471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ae5e-b279-4382-8cd8-75e75be00480 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:22.484951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-99084183-176f-41c3-b5a7-1b7583b308c0 None None] MFA Rules not processed for user `5db47a076f5d4650b0a7a6945ae70a52`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:22.491368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-99084183-176f-41c3-b5a7-1b7583b308c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.500347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:22.501617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 869/1770] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:22.505910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.505910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.505910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] PATH_INFO: `/v3/projects/8581d0c6d1394a2d8f37e073accd2630/users/c284ad25e4d04d40bc0afea5e951b4f7/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.539828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8581d0c6d1394a2d8f37e073accd2630, user_id=c284ad25e4d04d40bc0afea5e951b4f7, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:22.542712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:22.578233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 902/1771] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:22 2022] PUT /identity/v3/projects/8581d0c6d1394a2d8f37e073accd2630/users/c284ad25e4d04d40bc0afea5e951b4f7/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:22.588797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.589351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.589762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.604863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:22.606672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.682134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] MFA Rules not processed for user `c284ad25e4d04d40bc0afea5e951b4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:22.687939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:22.777403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c15e3-03f6-49ca-ab69-897759db8dcc admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:22.789319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:22.790052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:22.790345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:22.790623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:22.810839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:22.812710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:22.825361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 903/1772] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:23.014904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 870/1773] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:23.019105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.099881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.102568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.148824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d38460-8e57-4c27-8f68-552bd49615fc service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.153810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:23.154593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.154971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.155340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.156241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:23.158937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:23.203265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 904/1774] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:23 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:23.204334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.208110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.313384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa209bf-0721-45b4-b32c-761fae28fd68 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.320174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576f6aa-6e68-48ac-8300-5301b99d776d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.335577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:23.337492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.337978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.338473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] PATH_INFO: `/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.351635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b591a314bfc41fb92eab6ce05e7cd65)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:23.355731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:23.435785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:23.436803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.438045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.438466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.452435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b591a314bfc41fb92eab6ce05e7cd65', 'request_id': 'req-4f153086-02ee-475b-839e-ea5acd76d8a0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.461028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b591a314bfc41fb92eab6ce05e7cd65', 'request_id': 'req-4f153086-02ee-475b-839e-ea5acd76d8a0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.473854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b591a314bfc41fb92eab6ce05e7cd65', 'request_id': 'req-4f153086-02ee-475b-839e-ea5acd76d8a0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.498709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] The token cache is being invalidate because project 3b591a314bfc41fb92eab6ce05e7cd65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:23.501082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.502323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.505484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 905/1775] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:23 2022] DELETE /identity/v3/projects/3b591a314bfc41fb92eab6ce05e7cd65 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:23.507653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.512377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.601625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:23.603619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d120cce-3aa3-4a58-b690-280d6d713b5f service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:23.616489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153086-02ee-475b-839e-ea5acd76d8a0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.620611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 871/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:23.622182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.622608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.623057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.634128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:23.636395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.637971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.638369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.639185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:23.642287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44def16c-0e0b-4fc9-8a25-bf8cb7d530cf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:23.651539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:23.653199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.663817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 872/1777] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:23.664943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.667667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.693909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 906/1778] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:23 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:23.696841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73743ad8-761c-483d-ad02-da940e0ced7d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.697149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ee46f-a795-4299-9908-dae9be078b2e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.698064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ee46f-a795-4299-9908-dae9be078b2e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.698621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ee46f-a795-4299-9908-dae9be078b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.709679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:23.720282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.720757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.720757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] PATH_INFO: `/v3/projects/9eaedb848aec4e2d81bc106e894acbd5` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.722820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eaedb848aec4e2d81bc106e894acbd5)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:23.725154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:23.764494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1c3ee46f-a795-4299-9908-dae9be078b2e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:23.766925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1c3ee46f-a795-4299-9908-dae9be078b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.786986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eaedb848aec4e2d81bc106e894acbd5', 'request_id': 'req-fd2f2270-ff47-4364-8219-ef87e3765957'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.795509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eaedb848aec4e2d81bc106e894acbd5', 'request_id': 'req-fd2f2270-ff47-4364-8219-ef87e3765957'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.809538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eaedb848aec4e2d81bc106e894acbd5', 'request_id': 'req-fd2f2270-ff47-4364-8219-ef87e3765957'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.834215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] The token cache is being invalidate because project 9eaedb848aec4e2d81bc106e894acbd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:23.834751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.836707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:23.838821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 873/1779] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:23 2022] DELETE /identity/v3/projects/9eaedb848aec4e2d81bc106e894acbd5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:23.840148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.841803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.912744 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 907/1780] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:23.913626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:23.914009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:23.914374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:23.950817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:23.952063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:23.975206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 908/1781] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:23.976097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:23.990758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a265d50-f9a5-4e58-8f50-fa6ce72cecf0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:23.997628 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.001361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.001361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.002189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.003633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:24.003946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.005925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5546ed-5e35-4acb-abc5-73930c0ed061 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.025492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 909/1782] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:24.027013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.027303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.027959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.029690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f2270-ff47-4364-8219-ef87e3765957 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.046960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.051564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.054304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.054744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.058954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.060874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebb010-475a-44e7-b623-4ff15d1e8bec admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.081090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:24.082911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:24.095257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 910/1783] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:24.097938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 874/1784] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:24.098532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.102316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.102576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.102865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.114732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a43ac-030a-48f2-89fa-9449cf46a11d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.124084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.125084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.125510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.125952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.127104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.129504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.164687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] MFA Rules not processed for user `a71d4761b81447a194e2141442ceef1d`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:24.165978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:24.181324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 911/1785] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:24.182544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.301341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 875/1786] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:24.304470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.314780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d8b4-7b95-430d-9e67-911cb773acd2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.318651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185d5c7-2457-4e64-beea-dac9af617610 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.319818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.321321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.322645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.323047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] PATH_INFO: `/v3/projects/ef115a1de03e44fe95d4d0947c3c915c/users/008f601d3b78465dac5dc8286a30bbb0/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.325083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.325947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.326366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.326782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.327661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:24.328284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.330452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.345054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 876/1787] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:24.346521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.356039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef115a1de03e44fe95d4d0947c3c915c, user_id=008f601d3b78465dac5dc8286a30bbb0, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.359605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.369954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7691a00-a7f0-40c3-8c56-730b3d5a668c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.378800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.379352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.380182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.380182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.380616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.385478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.386621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 912/1788] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:24 2022] PUT /identity/v3/projects/ef115a1de03e44fe95d4d0947c3c915c/users/008f601d3b78465dac5dc8286a30bbb0/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:24.389424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.428624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 877/1789] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/users => generated 282 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:24.430927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.454789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753582-d551-4530-a6b1-f75cc81f44bc admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.481036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbac4c7-0d6f-4a49-84c1-8d67a38142af admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.494069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.496488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.499041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.499645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.503595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.514143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.587033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 913/1790] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:24.587866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.616693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.617399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.617678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.617954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.618694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.621128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.656758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 878/1791] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:24.657997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.675382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6734e-774e-4e28-be7d-8e44a46960ab admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.696762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81822c0e-b03f-48f5-ba44-d4e35dba7f27 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.702960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.710886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.711121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.711431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.712385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.714031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.728675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 914/1792] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:24.730479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.751734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab36eb0-89bb-4da6-b118-9e6a0f096278 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:24.777983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.779908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.782052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.782769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] PATH_INFO: `/v3/projects/ef115a1de03e44fe95d4d0947c3c915c/users/008f601d3b78465dac5dc8286a30bbb0/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.785513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef115a1de03e44fe95d4d0947c3c915c, user_id=008f601d3b78465dac5dc8286a30bbb0, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.801679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.808965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:24.809693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.810009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.810327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.811775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:24.815675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159be2d-8971-404f-8514-97203964fe0f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:24.825497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 915/1793] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:24 2022] PUT /identity/v3/projects/ef115a1de03e44fe95d4d0947c3c915c/users/008f601d3b78465dac5dc8286a30bbb0/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:24.827763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:24.866157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 879/1794] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:24.869163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:24.869448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:24.869728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:24.947720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] MFA Rules not processed for user `008f601d3b78465dac5dc8286a30bbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:24.949913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:24.997875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1031c-a58c-46c6-99fc-8ba9db209224 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.009021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.009726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.010009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.010309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.011110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.013849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.075422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 916/1795] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:25.076361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.100288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717905fe-9515-4c22-b7c9-12214caffdb0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.116511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 880/1796] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:25.117747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.125080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.126486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.126769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.127046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.127923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.132530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.138338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3f190-582e-4984-b8c8-148af4347714 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.160274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 917/1797] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:25.161137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.167570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:25.181594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.185221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.185528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.186882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] PATH_INFO: `/v3/projects/625d9e9286a04fdb98aa0f5c64bbe3ff/users/9709191200ea4fdfbc2b47110821f628/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.232709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=625d9e9286a04fdb98aa0f5c64bbe3ff, user_id=9709191200ea4fdfbc2b47110821f628, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.239113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.269066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 881/1798] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:25 2022] PUT /identity/v3/projects/625d9e9286a04fdb98aa0f5c64bbe3ff/users/9709191200ea4fdfbc2b47110821f628/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:25.270130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.279954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4c081-74c9-40ab-9b71-1db30d8279df admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.401338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.402322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.402669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.402948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.458791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57bcee-f10e-461d-9b0b-2f3348abb99d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.477964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.479911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.480992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.482023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.482935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.486206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fbfc4-58a9-470d-9858-e22af45ec999 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.512005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.513509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7570cc-8a11-4822-8c0f-91a63623efbc service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.529379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 882/1799] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:25.531630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.538196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.538493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.539561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 918/1800] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:25.544416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.545006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.546107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.562205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] MFA Rules not processed for user `5db47a076f5d4650b0a7a6945ae70a52`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:25.564067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:25.621865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] MFA Rules not processed for user `9709191200ea4fdfbc2b47110821f628`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:25.623504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:25.651201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 883/1801] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:25.652473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.674682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b78d3-751f-4cb8-ab98-9f89103eae0c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.679798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:25.680811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.681096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.681361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] PATH_INFO: `/v3/projects/e1b7caa872a0401ab68483beb5a39909/users/d1a25ed51bb6477ea317b9c4a078297e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.723580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b7caa872a0401ab68483beb5a39909, user_id=d1a25ed51bb6477ea317b9c4a078297e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:25.752779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122469c1-18cd-4d08-a76f-fc55bd77c80b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:25.793580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 884/1802] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:25 2022] PUT /identity/v3/projects/e1b7caa872a0401ab68483beb5a39909/users/d1a25ed51bb6477ea317b9c4a078297e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:25.799225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:25.802045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:25.802464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:25.848957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] MFA Rules not processed for user `d1a25ed51bb6477ea317b9c4a078297e`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:25.849509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:25.866016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 919/1803] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:25.870794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:25.911398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06bf9-8a1d-410c-a0fe-adffd78e70c8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:25.952856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 885/1804] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:25.954079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.076806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053e5c3-4467-4e4f-99bf-81b03bb77dcf None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.085184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.085424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.086948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.087257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.088179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.089140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.089862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.090245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.090618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.091326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.091877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.112434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.157463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 886/1805] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:25 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:26.158450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.210443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 920/1806] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:25 2022] POST /identity/v3/projects => generated 367 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:26.214725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.345885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982fd71-06bf-4263-b3df-7dbe89c91214 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.447081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746701-e752-4473-a618-b46cc36c63fa admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.466651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.467365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.467649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.467928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.468702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.476081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.525563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.527751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.527980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.532548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.532548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.532961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.534383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 921/1807] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:26 2022] POST /identity/v3/users => generated 290 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:26.535565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.561991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190a4cd-0403-439e-8eb9-6f4158096ecf admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.571499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.574394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.577406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.577891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.580588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.587305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.611813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 887/1808] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:26 2022] POST /identity/v3/users => generated 293 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:26.617961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.626085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 922/1809] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:26.627454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.653694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da798c-2d6f-4824-b9f2-d3ff32a53e3a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.663829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4861a-9f3e-4817-a3e5-a8d72c26051c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.668158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.668793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.669072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.669427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.670164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.682554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.683338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.683622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.683901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] PATH_INFO: `/v3/projects/bbed7af92f384296aff9f90def50bf6d/users/5ec724e7ee104660894c8cb3f63af83f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.685428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.718937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 888/1810] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:26.735950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbed7af92f384296aff9f90def50bf6d, user_id=5ec724e7ee104660894c8cb3f63af83f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.741476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.745770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.779593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 923/1811] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:26 2022] PUT /identity/v3/projects/bbed7af92f384296aff9f90def50bf6d/users/5ec724e7ee104660894c8cb3f63af83f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:26.784720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:26.785396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219ed68-118d-4a99-9344-1995496c8d2d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.959198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cae20-d8c2-4f42-8570-2d6d4c2ff3fe admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:26.965165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.966062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.966575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.967060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.968096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.970161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.989360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 924/1812] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:26.990232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:26.990999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:26.991248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:26.991528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:26.992407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:26.994012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:26.994300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:27.024645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 889/1813] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:27.033047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:27.039466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90ddbc-1b94-443d-aa25-e125369c485f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:27.048294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:27.051450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.051910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.052573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] PATH_INFO: `/v3/projects/bbed7af92f384296aff9f90def50bf6d/users/5ec724e7ee104660894c8cb3f63af83f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.059080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbed7af92f384296aff9f90def50bf6d, user_id=5ec724e7ee104660894c8cb3f63af83f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:27.079494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57baf4-6aa5-4fc1-90ad-faa19c499e71 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:27.079859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65398d-5998-4831-a3cc-ebf1108dc6e7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:27.090897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:27.094890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.095182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.095468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] PATH_INFO: `/v3/projects/a18819a38ae54ea99a329f3924a93008/users/b1ba2db0302e466db38e047d48480d2f/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.099385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 925/1814] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:26 2022] PUT /identity/v3/projects/bbed7af92f384296aff9f90def50bf6d/users/5ec724e7ee104660894c8cb3f63af83f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:27.114731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.115378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.115700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.143333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18819a38ae54ea99a329f3924a93008, user_id=b1ba2db0302e466db38e047d48480d2f, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:27.146321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114b4e0-d24f-4244-b2e1-ee9669f63ff4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:27.157946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] MFA Rules not processed for user `5ec724e7ee104660894c8cb3f63af83f`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:27.160070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:27.183967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 890/1815] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:27 2022] PUT /identity/v3/projects/a18819a38ae54ea99a329f3924a93008/users/b1ba2db0302e466db38e047d48480d2f/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:27.195630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.195917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.196195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.278557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] MFA Rules not processed for user `b1ba2db0302e466db38e047d48480d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:27.280107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:27.297966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 926/1816] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:27.311340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:27.456138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534f9a4-ca93-4f42-9476-33b73b278dfe None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:27.473710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 891/1817] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:27.481325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:27.605502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7c019-c0fe-4e6c-bea9-957c488de4f2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:27.615424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:27.616016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.616316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.616504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.625542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:27.626315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.626641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.626915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.627704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:27.629997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fbca1-73ca-4989-8671-5c0866a52a15 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:27.707286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 892/1818] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:27 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:27.710771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.713566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.714026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.751938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] MFA Rules not processed for user `6a41fc2b6e1c4032bfa408eaf36887ee`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:27.753211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:27.766550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:27.774506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:27.786834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 927/1819] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:27.787711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:27.810763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5af97-2de1-4981-8c54-3443e4159f41 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:27.955787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:27.956511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:27.956838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:27.957045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:27.957736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:27.959720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:27.976224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 893/1820] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:27.999466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 928/1821] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:27 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:28.003083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.022659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f9c6-c7bb-42f4-a696-f94252fc68f2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.023631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.039748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.041755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.043246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.048161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.049222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.051123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:28.089183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 894/1822] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:28.092567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.126839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7681f-55b2-486b-b852-441ed5d85cde admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.151865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.152463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.152870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.153909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] PATH_INFO: `/v3/projects/99e917f73d5c4eeaa8478e5637c22603/users/65f82dbe2e2d469085f6b5b901579f7e/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.165447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681319cc-56a2-4d78-9e56-0862f2247c45 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.169357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.170017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.170297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.170690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.181309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e917f73d5c4eeaa8478e5637c22603, user_id=65f82dbe2e2d469085f6b5b901579f7e, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.184012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:28.212329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 895/1823] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:28 2022] PUT /identity/v3/projects/99e917f73d5c4eeaa8478e5637c22603/users/65f82dbe2e2d469085f6b5b901579f7e/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:28.214337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.302763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.304519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:28.314576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 929/1824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:28.341179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38896d7-bdb9-45fe-a08c-ae91ddedf3dd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.350262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.351159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.351552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.352009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.353302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.357065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f36a25-bd94-41db-bf75-10f176c959a1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:28.372517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 896/1825] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:28.376691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.392795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f84e5-a8a3-4519-a5ae-37fb8c66afcb service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.403873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.406667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.407446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.408622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] PATH_INFO: `/v3/projects/99e917f73d5c4eeaa8478e5637c22603/users/65f82dbe2e2d469085f6b5b901579f7e/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.415539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e917f73d5c4eeaa8478e5637c22603, user_id=65f82dbe2e2d469085f6b5b901579f7e, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.422190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:28.459102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 930/1826] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:28 2022] PUT /identity/v3/projects/99e917f73d5c4eeaa8478e5637c22603/users/65f82dbe2e2d469085f6b5b901579f7e/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:28.462871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.463382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.464861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.517911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] MFA Rules not processed for user `65f82dbe2e2d469085f6b5b901579f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:28.519320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:28.673819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 897/1827] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:28.715457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.725309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:28.726871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:28.954889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051566f-5740-4ce0-ad30-a67a6da25329 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.955752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d3b43-47da-411b-90bb-d9cdd36c94da admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:28.970017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.971106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.971533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.972546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] PATH_INFO: `/v3/users/c284ad25e4d04d40bc0afea5e951b4f7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.973068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:28.974821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:28.975096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:28.975374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:28.980651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c284ad25e4d04d40bc0afea5e951b4f7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:28.982314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:29.017963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:29.019550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaae6c1-1d34-41d1-ac34-58aa62667174 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:29.059538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 898/1828] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:29.166290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c284ad25e4d04d40bc0afea5e951b4f7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:29.241987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c284ad25e4d04d40bc0afea5e951b4f7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:29.247003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c284ad25e4d04d40bc0afea5e951b4f7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:29.249075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c284ad25e4d04d40bc0afea5e951b4f7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:29.283706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c284ad25e4d04d40bc0afea5e951b4f7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:29.347240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 931/1829] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:28 2022] DELETE /identity/v3/users/c284ad25e4d04d40bc0afea5e951b4f7 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:29.352507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:29.354803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:29.355922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:29.423868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:29.426819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:29.553713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 899/1830] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:29.616118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:29.625281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:29.729387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc8632-7ed5-407e-bdef-6f77f755ba9f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:29.735119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:29.735998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:29.736489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:29.737316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:29.751889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:29.753748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20648bfd-c66e-4681-862b-e4ecfb928e16 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:29.769270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 932/1831] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:30.140432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:30.143084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:30.310440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a10e5a-c133-4a16-bd7b-ba2cbf20d6c2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:30.322854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:30.324959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:30.325249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:30.325531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:30.334809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:30.469672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4be449-5a3a-4d67-b406-eb76a457afa3 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:30.470227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4be449-5a3a-4d67-b406-eb76a457afa3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:30.470533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4be449-5a3a-4d67-b406-eb76a457afa3 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:30.472207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 933/1832] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:53:30.484448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:30.485715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:30.486007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:30.486290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:30.487479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:30.522763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:30.527186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:30.546840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 900/1833] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:30.731228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 934/1834] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:53:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:31.501961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:31.504299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:31.519084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65de04-4be5-40fc-85c9-db499c0fd7a7 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:31.524634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:31.525900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:31.526387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:31.527025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] PATH_INFO: `/v3/projects/8581d0c6d1394a2d8f37e073accd2630` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:31.529511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8581d0c6d1394a2d8f37e073accd2630)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:31.532282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:31.612322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8581d0c6d1394a2d8f37e073accd2630', 'request_id': 'req-b00c9649-7a94-4464-8b3b-5bd45ba44b75'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.622177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8581d0c6d1394a2d8f37e073accd2630', 'request_id': 'req-b00c9649-7a94-4464-8b3b-5bd45ba44b75'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.641195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8581d0c6d1394a2d8f37e073accd2630', 'request_id': 'req-b00c9649-7a94-4464-8b3b-5bd45ba44b75'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.665955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] The token cache is being invalidate because project 8581d0c6d1394a2d8f37e073accd2630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:31.666264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.667326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.674072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 901/1835] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:31 2022] DELETE /identity/v3/projects/8581d0c6d1394a2d8f37e073accd2630 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:31.674453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:31.686382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f20c-2ff0-4dd3-830e-fa6fdf28f074 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:31.762117 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:31.763719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:31.764150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:31.764594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] PATH_INFO: `/v3/users/008f601d3b78465dac5dc8286a30bbb0` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:31.766635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008f601d3b78465dac5dc8286a30bbb0)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:31.769003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:31.840960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008f601d3b78465dac5dc8286a30bbb0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.857032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008f601d3b78465dac5dc8286a30bbb0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.857960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008f601d3b78465dac5dc8286a30bbb0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.858623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008f601d3b78465dac5dc8286a30bbb0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.868291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bb76695d-af93-454c-9fa3-4a965a7619d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008f601d3b78465dac5dc8286a30bbb0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:31.918520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 935/1836] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:31 2022] DELETE /identity/v3/users/008f601d3b78465dac5dc8286a30bbb0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:32.162501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:32.164540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:32.403211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c9649-7a94-4464-8b3b-5bd45ba44b75 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:32.478608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:32.479184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:32.480003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:32.480447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:32.482660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:32.501313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:32.503371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864362e8-1e1a-4b28-ab75-e75b827b659c service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:32.516176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 902/1837] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:33.060417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:33.061453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:33.061729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:33.063676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 936/1838] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:53:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:53:33.106987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:33.107524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:33.108060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:33.147851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:33.152736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:33.182043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 903/1839] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:33.541028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:33.543670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:33.580155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5862c-d3bd-4d0f-879f-bf5b62e2d174 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:33.598869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:33.600613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:33.600907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:33.601997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:33.620875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:33.622621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f606d-19fc-499e-abdd-d9d79dd934a1 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:33.638772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 937/1840] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:34.205122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:34.206637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:34.386343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf0316-cbab-499c-a91a-5ba054658c20 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:34.391883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:34.393154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:34.393872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:34.394397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] PATH_INFO: `/v3/projects/ef115a1de03e44fe95d4d0947c3c915c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:34.395989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef115a1de03e44fe95d4d0947c3c915c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:34.398673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:34.538334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef115a1de03e44fe95d4d0947c3c915c', 'request_id': 'req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:34.549897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef115a1de03e44fe95d4d0947c3c915c', 'request_id': 'req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:34.563704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef115a1de03e44fe95d4d0947c3c915c', 'request_id': 'req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:34.577011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] The token cache is being invalidate because project ef115a1de03e44fe95d4d0947c3c915c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:34.577944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:34.578781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b950c8d0-12a4-466f-a2f2-f08eb3ade9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:34.582027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 904/1841] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:34 2022] DELETE /identity/v3/projects/ef115a1de03e44fe95d4d0947c3c915c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:34.586872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:34.587409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:34.587409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:34.625452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:34.626710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:34.719664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 938/1842] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:34.721713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:34.722645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:34.723491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:34.747736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:34.749337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:34.760590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 905/1843] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:34.764547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:34.776989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef6da2-a8e7-4a58-ae8d-fac45ee19577 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:34.782244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:34.783024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:34.783405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:34.783775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:34.784661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:34.785067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:34.786759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:34.800146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 939/1844] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:53:34.804542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:34.814598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e39f66-0b36-41e2-be38-2f13f1992486 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:34.819821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:34.820652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:34.821221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:34.821389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:34.822910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:34.825164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:34.889719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 906/1845] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:34 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:34.901882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:34.917759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cdab7-0473-47bd-b3b8-52e258d04387 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.103104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.103812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.104092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.104397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.105102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.111458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.172668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 940/1846] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:34 2022] POST /identity/v3/users => generated 292 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:35.175775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.195208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddc42-dd66-4f70-a804-bd85da75ee64 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.206111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.208682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.209182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.210016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.210443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.210884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.212589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.215156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.229462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 907/1847] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:35.233360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.247134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37afbe9-d887-403f-88cd-9e37209c07c3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.251099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcceaa-0927-4af8-84aa-b423c90dcd5a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.260053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.261414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.261849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.262270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.263182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.263482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.264422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.264696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.265455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] PATH_INFO: `/v3/users/9709191200ea4fdfbc2b47110821f628` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.266846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.269198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9709191200ea4fdfbc2b47110821f628)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.271326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.281194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 908/1848] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:35.285186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.319949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ad5ee-18ab-4bb1-aaa0-4d438f2bbc5c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.331146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.332219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.332705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.333323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] PATH_INFO: `/v3/projects/85eb99f080dd462dbe3b59e494ea713b/users/10d40d235b9d44e583204449006f5ba7/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.355770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eb99f080dd462dbe3b59e494ea713b, user_id=10d40d235b9d44e583204449006f5ba7, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.359074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d0b3c-0d8b-404e-aa72-551d695a63a3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.361714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9709191200ea4fdfbc2b47110821f628'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:35.380484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9709191200ea4fdfbc2b47110821f628'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:35.382702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9709191200ea4fdfbc2b47110821f628'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:35.383484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9709191200ea4fdfbc2b47110821f628'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:35.406011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 909/1849] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:35 2022] PUT /identity/v3/projects/85eb99f080dd462dbe3b59e494ea713b/users/10d40d235b9d44e583204449006f5ba7/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:35.413928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-051768f8-f9a3-47ed-a9cf-76bc0c5fbd39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9709191200ea4fdfbc2b47110821f628'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:35.417898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.418436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.419193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.436584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 941/1850] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:35 2022] DELETE /identity/v3/users/9709191200ea4fdfbc2b47110821f628 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:35.441780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.442307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.442850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.469301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:35.471242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.488297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] MFA Rules not processed for user `10d40d235b9d44e583204449006f5ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:35.490499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.608020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 942/1851] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:35.634358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 910/1852] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:35.638816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.640436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.656218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf448b-a1de-45b3-8a55-baa25ec3886f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.662589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.663424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.663707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.663985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.665293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.669794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.722739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 943/1853] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:35 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:35.726461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.741783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833302c-4eb4-4af9-a502-966e88ce4a6c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.782639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.784706 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.820357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.822026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.822657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.823386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.824223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.826536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.854556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 911/1854] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:35 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:35.854556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.859706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.879309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac7e-03fb-4d25-8566-0d913aa10519 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.883554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57732915-128a-464a-bebc-595719666c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.884517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732915-128a-464a-bebc-595719666c30 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.885055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732915-128a-464a-bebc-595719666c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.885513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732915-128a-464a-bebc-595719666c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.888699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-57732915-128a-464a-bebc-595719666c30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:35.959360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348501d-57ea-4b5e-a908-5c07010f0802 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:35.970160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:35.970804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57732915-128a-464a-bebc-595719666c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.971077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:35.971460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:35.971839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:35.972379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57732915-128a-464a-bebc-595719666c30 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.983723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 944/1855] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:35.985859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57732915-128a-464a-bebc-595719666c30 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:35.994105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:35.997942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:35.999666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57732915-128a-464a-bebc-595719666c30 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.005980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.007315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.007671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.008056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.009864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.011453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.023045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 912/1856] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:36.026395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 945/1857] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:36.031584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:36.045802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560bb6-6129-430f-a0c5-efbaf3e54588 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.051748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.052782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.053336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.053702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] PATH_INFO: `/v3/projects/3351c8f3d3b3493a8febce33486f36da/users/588936ccef03452c8e6659ce07e2ccdf/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.105023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3351c8f3d3b3493a8febce33486f36da, user_id=588936ccef03452c8e6659ce07e2ccdf, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.110576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.143401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 913/1858] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:36 2022] PUT /identity/v3/projects/3351c8f3d3b3493a8febce33486f36da/users/588936ccef03452c8e6659ce07e2ccdf/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:36.149193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:36.173750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e1afe-67e3-4339-98c2-8ca05bf18d30 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.300383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.301863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.302166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.302466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.304050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.306364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9b528-95c7-4684-93f9-0a22db7d82de admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.348608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 946/1859] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:36.349812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:36.365461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5e29-72ef-47f2-b8a8-8804df479a1c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.413212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.424666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.425135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.425555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] PATH_INFO: `/v3/projects/3351c8f3d3b3493a8febce33486f36da/users/588936ccef03452c8e6659ce07e2ccdf/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.427898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3351c8f3d3b3493a8febce33486f36da, user_id=588936ccef03452c8e6659ce07e2ccdf, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.431473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.445676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 914/1860] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:36 2022] PUT /identity/v3/projects/3351c8f3d3b3493a8febce33486f36da/users/588936ccef03452c8e6659ce07e2ccdf/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:36.455995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.456322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.456608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.511926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] MFA Rules not processed for user `588936ccef03452c8e6659ce07e2ccdf`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:36.513194 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:36.645356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 947/1861] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:36.658602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:36.661555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:36.674013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:36.675571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:36.795815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a930-5334-4920-b72e-20c24c4fbaca None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.803052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.805392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.805899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.806466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] PATH_INFO: `/v3/projects/625d9e9286a04fdb98aa0f5c64bbe3ff` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.808733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625d9e9286a04fdb98aa0f5c64bbe3ff)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.811527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.833250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e52e7-2e3b-44ad-a394-c9cef028d987 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:36.843564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:36.843564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:36.843819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:36.844310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:36.864998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:36.876352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:36.908446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 915/1862] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:36.952853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625d9e9286a04fdb98aa0f5c64bbe3ff', 'request_id': 'req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:36.960579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625d9e9286a04fdb98aa0f5c64bbe3ff', 'request_id': 'req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:36.973186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625d9e9286a04fdb98aa0f5c64bbe3ff', 'request_id': 'req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.006206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] The token cache is being invalidate because project 625d9e9286a04fdb98aa0f5c64bbe3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:37.008051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.010337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.022139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 948/1863] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:36 2022] DELETE /identity/v3/projects/625d9e9286a04fdb98aa0f5c64bbe3ff => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:37.023773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:37.051599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a534fef-d73c-40a6-9adc-9a7100e833e2 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:37.115637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:37.117107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:37.243213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:37.243740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:37.244018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:37.244352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] PATH_INFO: `/v3/users/5ec724e7ee104660894c8cb3f63af83f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:37.246506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec724e7ee104660894c8cb3f63af83f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:37.249847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f56fdb-33ad-4316-8eb3-9da4d70b9ab0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:37.250057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:37.255391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:37.256384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:37.256783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:37.257125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:37.261037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:37.356641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:37.358650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:37.366252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec724e7ee104660894c8cb3f63af83f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.370477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 949/1864] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:37.450195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec724e7ee104660894c8cb3f63af83f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.466593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec724e7ee104660894c8cb3f63af83f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.475045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec724e7ee104660894c8cb3f63af83f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.489290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-372f27c9-7d8b-4f7f-907a-9bcb1fe32926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec724e7ee104660894c8cb3f63af83f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:37.514669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 916/1865] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:37 2022] DELETE /identity/v3/users/5ec724e7ee104660894c8cb3f63af83f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:37.917995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:37.919469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:37.942674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc1dc-dec0-4c5d-85dc-99af4f188789 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:38.049179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:38.050253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:38.050686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:38.051059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] PATH_INFO: `/v3/users/10d40d235b9d44e583204449006f5ba7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:38.052510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d40d235b9d44e583204449006f5ba7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:38.054822 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:38.119168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d40d235b9d44e583204449006f5ba7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:38.141979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d40d235b9d44e583204449006f5ba7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:38.142742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d40d235b9d44e583204449006f5ba7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:38.143400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d40d235b9d44e583204449006f5ba7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:38.162659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d40d235b9d44e583204449006f5ba7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:38.189010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 950/1866] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:37 2022] DELETE /identity/v3/users/10d40d235b9d44e583204449006f5ba7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:38.194280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:38.194573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:38.194851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:38.221445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:38.224483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:38.309698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 917/1867] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:38.343573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:38.346906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:38.436441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa260b2e-9adf-4438-854e-5c85619c7195 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:38.440764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:38.441533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:38.441886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:38.442234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:38.453275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:38.455985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:38.481013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 951/1868] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:39.182025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.183558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.207154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e492df-0072-41a0-957a-5a71cbf83d28 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:39.221043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:39.222228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.222502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.222787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] PATH_INFO: `/v3/projects/bbed7af92f384296aff9f90def50bf6d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.225721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbed7af92f384296aff9f90def50bf6d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:39.227722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:39.351454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbed7af92f384296aff9f90def50bf6d', 'request_id': 'req-0e09b6cf-75a8-427b-bf8f-641e2613ac12'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.371717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbed7af92f384296aff9f90def50bf6d', 'request_id': 'req-0e09b6cf-75a8-427b-bf8f-641e2613ac12'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.400553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbed7af92f384296aff9f90def50bf6d', 'request_id': 'req-0e09b6cf-75a8-427b-bf8f-641e2613ac12'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.443550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] The token cache is being invalidate because project bbed7af92f384296aff9f90def50bf6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:39.444126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.445875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.447633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0e09b6cf-75a8-427b-bf8f-641e2613ac12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.448665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.455225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 918/1869] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:39 2022] DELETE /identity/v3/projects/bbed7af92f384296aff9f90def50bf6d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:39.460089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.460421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.460774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.471848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2329ae-867a-4501-b4ef-d44c85a6bf03 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:39.492758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:39.494402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.588679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:39.590384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.591019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.591520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] PATH_INFO: `/v3/users/d1a25ed51bb6477ea317b9c4a078297e` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.593217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a25ed51bb6477ea317b9c4a078297e)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:39.596435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:39.606340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 919/1870] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:39.606340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.610435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.636804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689872e-a311-42f0-8f06-b1d201d1721f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:39.647016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:39.648005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.648331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.648610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] PATH_INFO: `/v3/projects/85eb99f080dd462dbe3b59e494ea713b` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.649953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] RBAC: Authorizing `identity:delete_project(project_id=85eb99f080dd462dbe3b59e494ea713b)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:39.652719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:39.697930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a25ed51bb6477ea317b9c4a078297e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.718484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a25ed51bb6477ea317b9c4a078297e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.719408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a25ed51bb6477ea317b9c4a078297e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.720156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a25ed51bb6477ea317b9c4a078297e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.726065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85eb99f080dd462dbe3b59e494ea713b', 'request_id': 'req-879e4269-0fd4-46de-9671-50756bb3b1be'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.727286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d03a8c9f-0572-46b9-ba5c-68b8e552df0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a25ed51bb6477ea317b9c4a078297e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.738348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85eb99f080dd462dbe3b59e494ea713b', 'request_id': 'req-879e4269-0fd4-46de-9671-50756bb3b1be'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.745567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 952/1871] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:39 2022] DELETE /identity/v3/users/d1a25ed51bb6477ea317b9c4a078297e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:39.747029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.747315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.748873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.750948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85eb99f080dd462dbe3b59e494ea713b', 'request_id': 'req-879e4269-0fd4-46de-9671-50756bb3b1be'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.763777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] The token cache is being invalidate because project 85eb99f080dd462dbe3b59e494ea713b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:39.764074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.764795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-879e4269-0fd4-46de-9671-50756bb3b1be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.767200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 920/1872] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:39 2022] DELETE /identity/v3/projects/85eb99f080dd462dbe3b59e494ea713b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:39.767563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:39.768766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.768875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.769055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.769336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.799295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:39.800550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.827999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 953/1873] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:39.829012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.830258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.841209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b538c6-84e2-49bd-b9df-1b40b5a9b249 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:39.846454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:39.847326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.847642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.847894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] PATH_INFO: `/v3/users/588936ccef03452c8e6659ce07e2ccdf` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.849197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=588936ccef03452c8e6659ce07e2ccdf)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:39.851129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:39.859112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 921/1874] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:39.860475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.872483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c786ad-446b-4a10-858d-e4ad60f2b8f3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:39.878505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:39.879402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:39.879686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:39.879992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:39.880898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:39.881216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:39.882836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:39.900792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 922/1875] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:39.900792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.903319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.909894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588936ccef03452c8e6659ce07e2ccdf'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.925663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588936ccef03452c8e6659ce07e2ccdf'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.926542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588936ccef03452c8e6659ce07e2ccdf'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.927638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588936ccef03452c8e6659ce07e2ccdf'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.934788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588936ccef03452c8e6659ce07e2ccdf'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:39.952295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 954/1876] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:39 2022] DELETE /identity/v3/users/588936ccef03452c8e6659ce07e2ccdf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:39.952295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:39.952295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:39.997232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fe763-1c99-4bd0-a3f7-25c1326983ae admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.073350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db963-5e27-4110-ab7a-bad30c54f72c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.077005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.078025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.079125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.079125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.081136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.085857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.087086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.088311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.088311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.089864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.095407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.098279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.099712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.121304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 923/1877] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:40.149078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 955/1878] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:39 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:40.153332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.166021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37b44e-e9ca-4844-8ff6-47f56390a3b9 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.242650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.244915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.245788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.246212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.248284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.252437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.277320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 924/1879] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:40 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:40.281998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.299531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcf732-e91e-4764-9e14-5db54b840d6a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.306772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.307458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.307791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.308065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.308837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.323299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.336379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 956/1880] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:40.339344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.350000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7698993-b473-4bd9-ac5b-eb41c7249b45 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.354596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.355287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.355545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.355816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.356588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.358233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.369002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 925/1881] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:40.373295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.383374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8b2bc-54dc-4ae5-ae80-2f8160fa1ce0 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.388659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.389672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.390030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.390364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] PATH_INFO: `/v3/projects/a28770065d7b4a20ba5ff05e1c881ede/users/5364eab1fe474c56b19bbf09d093fb7a/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.419436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28770065d7b4a20ba5ff05e1c881ede, user_id=5364eab1fe474c56b19bbf09d093fb7a, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.423235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df839-64fc-4e4b-99b8-52a59949ad8a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.436720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 957/1882] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:40 2022] PUT /identity/v3/projects/a28770065d7b4a20ba5ff05e1c881ede/users/5364eab1fe474c56b19bbf09d093fb7a/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:40.438348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.447793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.448555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.449314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.449885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.474878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56683c91-c489-41aa-b558-f869911fa0b9 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.481246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] MFA Rules not processed for user `5364eab1fe474c56b19bbf09d093fb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:40.483080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.545767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888489d4-3802-4910-850e-10441a06245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.546677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888489d4-3802-4910-850e-10441a06245f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.547161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888489d4-3802-4910-850e-10441a06245f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.547373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888489d4-3802-4910-850e-10441a06245f admin admin] PATH_INFO: `/v3/projects/3351c8f3d3b3493a8febce33486f36da` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.549692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888489d4-3802-4910-850e-10441a06245f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3351c8f3d3b3493a8febce33486f36da)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.551800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888489d4-3802-4910-850e-10441a06245f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.576471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 958/1883] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:40.590525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.592084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.618263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc4395-5f64-47bd-be93-91d0c36b8b0e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.639083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3351c8f3d3b3493a8febce33486f36da', 'request_id': 'req-888489d4-3802-4910-850e-10441a06245f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.645706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3351c8f3d3b3493a8febce33486f36da', 'request_id': 'req-888489d4-3802-4910-850e-10441a06245f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.654473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3351c8f3d3b3493a8febce33486f36da', 'request_id': 'req-888489d4-3802-4910-850e-10441a06245f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.667688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] The token cache is being invalidate because project 3351c8f3d3b3493a8febce33486f36da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:40.667688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.668558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.671932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 926/1884] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:40 2022] DELETE /identity/v3/projects/3351c8f3d3b3493a8febce33486f36da => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:40.672778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:40.674322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.692144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.693292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.693650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.695699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] PATH_INFO: `/v3/projects/e1b7caa872a0401ab68483beb5a39909` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.697094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b7caa872a0401ab68483beb5a39909)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.699226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.756048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b7caa872a0401ab68483beb5a39909', 'request_id': 'req-4fa9278c-d08f-453d-a13a-87fcd4a06a40'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.763936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b7caa872a0401ab68483beb5a39909', 'request_id': 'req-4fa9278c-d08f-453d-a13a-87fcd4a06a40'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.764727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888489d4-3802-4910-850e-10441a06245f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:40.798349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b7caa872a0401ab68483beb5a39909', 'request_id': 'req-4fa9278c-d08f-453d-a13a-87fcd4a06a40'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.814927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] The token cache is being invalidate because project e1b7caa872a0401ab68483beb5a39909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:40.815364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.816167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4fa9278c-d08f-453d-a13a-87fcd4a06a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:40.820860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 959/1885] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:40 2022] DELETE /identity/v3/projects/e1b7caa872a0401ab68483beb5a39909 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:40.825248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40628391-649e-446a-890f-38d2daa29233 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.825546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40628391-649e-446a-890f-38d2daa29233 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.825823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40628391-649e-446a-890f-38d2daa29233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.841342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:40.842010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:40.842289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:40.842562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:40.847035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.855610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-40628391-649e-446a-890f-38d2daa29233 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:40.856942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-40628391-649e-446a-890f-38d2daa29233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:40.980912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:40.984906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc3ae5-b57c-4d4b-8a6a-4655675bdc12 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:40.992840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 960/1886] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:40.999061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40628391-649e-446a-890f-38d2daa29233 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.000315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-40628391-649e-446a-890f-38d2daa29233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.006987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 927/1887] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:41.012545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.014117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.014500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.034750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40628391-649e-446a-890f-38d2daa29233 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.052388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.053564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.053973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.054297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] PATH_INFO: `/v3/users/b1ba2db0302e466db38e047d48480d2f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.057209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ba2db0302e466db38e047d48480d2f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.059669 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.066650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:41.077063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.100700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 928/1888] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:41.100700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.107432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.127953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299904b1-47c0-4ac3-9bc3-ab5d2430062b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.134448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.135896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.136192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.136497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] PATH_INFO: `/v3/users/5364eab1fe474c56b19bbf09d093fb7a` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.137964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5364eab1fe474c56b19bbf09d093fb7a)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.140088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.180478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ba2db0302e466db38e047d48480d2f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.204840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ba2db0302e466db38e047d48480d2f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.206007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ba2db0302e466db38e047d48480d2f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.206801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ba2db0302e466db38e047d48480d2f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.234667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ba2db0302e466db38e047d48480d2f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.260555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 961/1889] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:40 2022] DELETE /identity/v3/users/b1ba2db0302e466db38e047d48480d2f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:41.265177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.266784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.288852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5364eab1fe474c56b19bbf09d093fb7a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.320623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5364eab1fe474c56b19bbf09d093fb7a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.339021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5364eab1fe474c56b19bbf09d093fb7a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.343301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5364eab1fe474c56b19bbf09d093fb7a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.369559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1953d377-d973-4c50-b82c-e00f4f1c751c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5364eab1fe474c56b19bbf09d093fb7a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:41.387926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec215832-7a61-4de3-8731-c53f2ba8e4b4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.388533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 929/1890] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:41 2022] DELETE /identity/v3/users/5364eab1fe474c56b19bbf09d093fb7a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:41.394085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.394779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.394779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.432428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:41.434026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.478199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.478965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.479245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.479530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.480286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:41.480600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.482449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.498506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 962/1891] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:41.503579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.515551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.535547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e203e7-8a7c-4a01-ad0f-75173e66be7d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.544105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 930/1892] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:41.545754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.546758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.547186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.547609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.548480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.550492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.561935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.563708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:41.602074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 963/1893] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:41 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:41.605977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.684816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d04f3e-aab2-4dc8-8c82-c69e0572356f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.732689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5280f-42d9-41b1-8aec-fd157a828377 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.740962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.741152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.741398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.741716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.742426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.744458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.774649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 964/1894] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:41 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:41.779010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.797115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d60ac-2d19-4cab-b25e-a25a23facfc5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.800212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.801651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.802165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.802548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.802980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.803531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.804024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.804201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.805845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.807638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.817674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.819393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.820613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 965/1895] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:41.825145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.831417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 931/1896] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:41.837569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e94c1d-982c-4e2e-ba92-474fbf0d2058 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.843222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.843618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.844073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.844512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.845352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.847104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a5c70-5247-4108-900c-87659c0c05b0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.863005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 966/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:41.864211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:41.879117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd47648-84ed-46a4-88cf-327d7a86fc89 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:41.884469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:41.885366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.885642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.885910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] PATH_INFO: `/v3/projects/558dbce802604ea1b3e75e6eb707f7d4/users/37ab6357d1ba44bc863bcadc775323c3/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.905664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=558dbce802604ea1b3e75e6eb707f7d4, user_id=37ab6357d1ba44bc863bcadc775323c3, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:41.909385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:41.933621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 932/1898] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:41 2022] PUT /identity/v3/projects/558dbce802604ea1b3e75e6eb707f7d4/users/37ab6357d1ba44bc863bcadc775323c3/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:41.946255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:41.947223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:41.947385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:41.980444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] MFA Rules not processed for user `37ab6357d1ba44bc863bcadc775323c3`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:41.981765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.045638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.047554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.131169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 967/1899] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:42.157000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.159299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.185254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f6e14-2f0f-4356-8789-4fed951b37ad admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.192051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.193269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.193623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.194017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] PATH_INFO: `/v3/projects/a18819a38ae54ea99a329f3924a93008` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.195695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18819a38ae54ea99a329f3924a93008)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.198387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.293348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544333a8-cc0a-4dfe-98ee-52b11c149a52 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.298934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18819a38ae54ea99a329f3924a93008', 'request_id': 'req-e6390795-b166-4b01-a5d7-1366b44510bf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.308894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18819a38ae54ea99a329f3924a93008', 'request_id': 'req-e6390795-b166-4b01-a5d7-1366b44510bf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.322406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18819a38ae54ea99a329f3924a93008', 'request_id': 'req-e6390795-b166-4b01-a5d7-1366b44510bf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.345245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] The token cache is being invalidate because project a18819a38ae54ea99a329f3924a93008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:42.345547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.346628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.360270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 933/1900] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:42 2022] DELETE /identity/v3/projects/a18819a38ae54ea99a329f3924a93008 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:42.365597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.368532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.373725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.374788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.375162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.376051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.379333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.446355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.447802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.457534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 968/1901] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:42.471220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6390795-b166-4b01-a5d7-1366b44510bf admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.476703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.476703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.476703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.477055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] PATH_INFO: `/v3/users/65f82dbe2e2d469085f6b5b901579f7e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.478198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f82dbe2e2d469085f6b5b901579f7e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.478299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.479617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.481520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.495126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c3407-28bc-4f6e-a554-da094e929282 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.503928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.504879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.505162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.505448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] PATH_INFO: `/v3/projects/a28770065d7b4a20ba5ff05e1c881ede` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.506637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28770065d7b4a20ba5ff05e1c881ede)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.508584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.537452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f82dbe2e2d469085f6b5b901579f7e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.550530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f82dbe2e2d469085f6b5b901579f7e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.558054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f82dbe2e2d469085f6b5b901579f7e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.558785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f82dbe2e2d469085f6b5b901579f7e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.569530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f82dbe2e2d469085f6b5b901579f7e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.574985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28770065d7b4a20ba5ff05e1c881ede', 'request_id': 'req-2cab45dc-768a-49b4-81d0-b15fc9adbed7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.578567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 934/1902] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:42 2022] DELETE /identity/v3/users/65f82dbe2e2d469085f6b5b901579f7e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:42.578567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.579147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.585508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28770065d7b4a20ba5ff05e1c881ede', 'request_id': 'req-2cab45dc-768a-49b4-81d0-b15fc9adbed7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.595261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28770065d7b4a20ba5ff05e1c881ede', 'request_id': 'req-2cab45dc-768a-49b4-81d0-b15fc9adbed7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.606743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] The token cache is being invalidate because project a28770065d7b4a20ba5ff05e1c881ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:42.607030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.607692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2cab45dc-768a-49b4-81d0-b15fc9adbed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.613226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 969/1903] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:42 2022] DELETE /identity/v3/projects/a28770065d7b4a20ba5ff05e1c881ede => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:42.614972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dccd9-d743-4db0-9bd9-721cf93acde8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.615256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dccd9-d743-4db0-9bd9-721cf93acde8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.615539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dccd9-d743-4db0-9bd9-721cf93acde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.636653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-889dccd9-d743-4db0-9bd9-721cf93acde8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:42.638012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-889dccd9-d743-4db0-9bd9-721cf93acde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.657480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3bb09-9915-481d-9e0a-800c71cd2b4e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.663126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.664050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.664454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.664724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] PATH_INFO: `/v3/users/37ab6357d1ba44bc863bcadc775323c3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.665980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ab6357d1ba44bc863bcadc775323c3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.667916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.718413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 970/1904] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:42.720312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.720581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.720859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.721795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ab6357d1ba44bc863bcadc775323c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.739111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ab6357d1ba44bc863bcadc775323c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.741791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:42.743349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.749976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ab6357d1ba44bc863bcadc775323c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.760090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ab6357d1ba44bc863bcadc775323c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.760292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 971/1905] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:42.760292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.767404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9b4c0877-43d6-4960-9a91-3597c77c48d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ab6357d1ba44bc863bcadc775323c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:42.771271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46a981-ef20-45fd-aeed-93ac98a03df9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.774785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 935/1906] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:42 2022] DELETE /identity/v3/users/37ab6357d1ba44bc863bcadc775323c3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:42.779403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.780310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.780580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.780765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.781658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.781658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.782024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.782850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:42.783217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.785008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.799632 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 972/1907] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:42.801078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:42.802360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.806457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.808375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.820678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda02e93-ace9-4f7b-8228-cbfbc98c2443 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:42.902363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 936/1908] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:42.915195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.916410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:42.916737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:42.917812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:42.919452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:42.920027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:42.922845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:42.925775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:42.977920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 973/1909] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:42 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:42.985835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:42.992448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55150-1a96-4b10-a80e-1e5cbb738bbd None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.056748 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea14-8529-4a94-a405-5c0bf2fc189c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.060277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.060962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.061246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.061662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.062492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.063177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.063458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.063777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.064676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.066941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.072830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.074285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.085314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 937/1910] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:43.086081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.087813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.098333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 974/1911] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:42 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:43.102907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.105307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590205b-f9b2-4ced-987c-ba3ee040638d service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.113373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6114d-83e4-43b8-a85a-b7dcf26ac829 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.118869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.119540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.119818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.120090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.120855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.121346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.122308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.122453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.123211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.123562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] PATH_INFO: `/v3/projects/99e917f73d5c4eeaa8478e5637c22603` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.124987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e917f73d5c4eeaa8478e5637c22603)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.126977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.133869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 975/1912] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:43.137676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.153368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d0622-5d29-4ce5-a2b6-b177efde49be admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.159356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.160721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.161306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.161734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.162771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.164900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.178435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 976/1913] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:43.181701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.184645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e917f73d5c4eeaa8478e5637c22603', 'request_id': 'req-678fd75e-6fea-43ba-b8d0-de9610df86e4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.190961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e917f73d5c4eeaa8478e5637c22603', 'request_id': 'req-678fd75e-6fea-43ba-b8d0-de9610df86e4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.192994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9dfc-48a7-42fb-a130-49bc00712a48 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.201612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e917f73d5c4eeaa8478e5637c22603', 'request_id': 'req-678fd75e-6fea-43ba-b8d0-de9610df86e4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.214682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] The token cache is being invalidate because project 99e917f73d5c4eeaa8478e5637c22603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:43.214978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.215627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-678fd75e-6fea-43ba-b8d0-de9610df86e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.222299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 938/1914] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:43 2022] DELETE /identity/v3/projects/99e917f73d5c4eeaa8478e5637c22603 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:43.224039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f0cd2-e098-4676-b4b9-1e07dcf50c4e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.224513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f0cd2-e098-4676-b4b9-1e07dcf50c4e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.225190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f0cd2-e098-4676-b4b9-1e07dcf50c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.253326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-de5f0cd2-e098-4676-b4b9-1e07dcf50c4e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:43.254569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-de5f0cd2-e098-4676-b4b9-1e07dcf50c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.262118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.263031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.263311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.263591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] PATH_INFO: `/v3/projects/60176266845d46219a1493294f118e94/users/486c247a63e04e159b7d911e4239ed35/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.281802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60176266845d46219a1493294f118e94, user_id=486c247a63e04e159b7d911e4239ed35, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.284504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32a194-9905-4b92-aa9d-59dd6c40dbb4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.307380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 977/1915] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:43 2022] PUT /identity/v3/projects/60176266845d46219a1493294f118e94/users/486c247a63e04e159b7d911e4239ed35/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:43.318131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.318412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.318688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.346359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 939/1916] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:43.351636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] MFA Rules not processed for user `486c247a63e04e159b7d911e4239ed35`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:43.353040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.356656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.356946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.357231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.377300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:43.378513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.440298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 978/1917] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:43.452717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.455600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.458857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 940/1918] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:43.459870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.470743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe7d0-679d-46af-95cc-6bf6207571f8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.477680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.478916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.479429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.479892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.480954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:43.481593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.486035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.499962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 941/1919] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:43.504062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.506083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.519110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b675-5286-40d5-88a7-1291a2ac083e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.524764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.525827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.526745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.527032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] PATH_INFO: `/v3/projects/558dbce802604ea1b3e75e6eb707f7d4` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.529115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=558dbce802604ea1b3e75e6eb707f7d4)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.531175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.535799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ef990-46f2-4316-a6be-a3514c7120bb None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.540561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.541340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.544866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.545329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.556170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.557887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.567856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 979/1920] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 00:53:43.568840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.581656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9727-800e-4368-be5f-384388ddaac8 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.597523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '558dbce802604ea1b3e75e6eb707f7d4', 'request_id': 'req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.606248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '558dbce802604ea1b3e75e6eb707f7d4', 'request_id': 'req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.615897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '558dbce802604ea1b3e75e6eb707f7d4', 'request_id': 'req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.630289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] The token cache is being invalidate because project 558dbce802604ea1b3e75e6eb707f7d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:43.630577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.631228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:43.633191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 942/1921] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:43 2022] DELETE /identity/v3/projects/558dbce802604ea1b3e75e6eb707f7d4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:43.652542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.653233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.653513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.653815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.654529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.656735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.687283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 980/1922] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:43 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:43.692019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.693166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:43.706822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3c02d-42e3-4dfa-b017-0dd54f9a3ad5 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.770369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.771077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.771357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.771631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.773510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.776338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.812389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 943/1923] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:43 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:43.813380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.826494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c814-b4ef-469f-88ba-52ead92d6ee2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.834083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.835030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.835525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.835902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.836775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.838684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.860958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 981/1924] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:43.865427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.878272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e3520-a1fa-455e-a6d8-dceb3048be5a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.883547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.884177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.884518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.884756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.885492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.887293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f079b2-f0d2-4077-935d-fd567c8d711f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:43.908163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 944/1925] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:43.912036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:43.925189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f952f-4ade-4d2e-81c1-c9a75f64fdb1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:43.942288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:43.943219 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:43.943560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:43.943774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] PATH_INFO: `/v3/projects/f42d65687a174308a488fa60bf1c6d64/users/96c1c383e5734104a3e0631542ddf987/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:43.978040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42d65687a174308a488fa60bf1c6d64, user_id=96c1c383e5734104a3e0631542ddf987, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:43.980853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.007995 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 982/1926] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:43 2022] PUT /identity/v3/projects/f42d65687a174308a488fa60bf1c6d64/users/96c1c383e5734104a3e0631542ddf987/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:44.019796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.020045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.020400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.047211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] MFA Rules not processed for user `96c1c383e5734104a3e0631542ddf987`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:44.048470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:44.116219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 945/1927] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:44.117678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.194690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580f06-324d-4db9-b9ae-7d373ee9c04e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.199496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.200167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.200481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.200754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.201575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.203634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.260703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 983/1928] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:44 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:44.261849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.276847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a183e-8222-4064-b1a8-3a958ad63f9a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.352616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.353198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.353437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.353710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.354460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.356425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.379668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 946/1929] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:44 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:44.382207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.392304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1619c1-90a4-4136-aaaa-ada2e3f489e3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.396799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.397451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.397732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.398004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.398726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.400387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.410668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 984/1930] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:44.414692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.425151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944eca8-040b-4eb6-ac09-e46eb07cf8e2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.430535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.431433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.431713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.431985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] PATH_INFO: `/v3/projects/40d3411ca6e6421990a092f18143767a/users/b69be48d0d0b4902bae18265ce1d4ae5/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.447221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d3411ca6e6421990a092f18143767a, user_id=b69be48d0d0b4902bae18265ce1d4ae5, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.449902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.465217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 947/1931] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:44 2022] PUT /identity/v3/projects/40d3411ca6e6421990a092f18143767a/users/b69be48d0d0b4902bae18265ce1d4ae5/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:44.469667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.479392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9aa31-755a-424d-88c8-28516526d40e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.536096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.536769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.536985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.537254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.537979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.539827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2923d4e-1e4a-4e63-9210-92c78ff0b244 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.550187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 985/1932] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:44.554454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.565383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6b3dc-e77d-464f-8ae2-36bc21f7bdfb admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.569835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.570752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.571031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.571303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] PATH_INFO: `/v3/projects/40d3411ca6e6421990a092f18143767a/users/b69be48d0d0b4902bae18265ce1d4ae5/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.573307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d3411ca6e6421990a092f18143767a, user_id=b69be48d0d0b4902bae18265ce1d4ae5, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.575934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.587468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 948/1933] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:44 2022] PUT /identity/v3/projects/40d3411ca6e6421990a092f18143767a/users/b69be48d0d0b4902bae18265ce1d4ae5/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:44.598559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.598851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.599124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.627798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] MFA Rules not processed for user `b69be48d0d0b4902bae18265ce1d4ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:44.629112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:44.698882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 986/1934] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:44.711286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:44.713458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:44.774633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83943a2-6e68-4ae0-986a-e2a782b71ef6 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:44.779165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:44.779811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:44.780109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:44.780441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:44.790934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:44.792415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc530-3166-491d-a207-8a1fdbff9e81 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:44.802591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 949/1935] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:45.400382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:45.469671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a7137-036c-4d37-89b2-0edef7e411ff None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:45.474404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:45.475315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:45.475599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:45.475878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] PATH_INFO: `/v3/users/96c1c383e5734104a3e0631542ddf987` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:45.477172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c1c383e5734104a3e0631542ddf987)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:45.479093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:45.528092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c1c383e5734104a3e0631542ddf987'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:45.540142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c1c383e5734104a3e0631542ddf987'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:45.540991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c1c383e5734104a3e0631542ddf987'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:45.541737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c1c383e5734104a3e0631542ddf987'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:45.550611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c1c383e5734104a3e0631542ddf987'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:45.566799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 987/1936] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:45 2022] DELETE /identity/v3/users/96c1c383e5734104a3e0631542ddf987 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:45.571359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:45.571849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:45.571944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:45.589990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:45.591201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:45.657379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 950/1937] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:45.667215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:45.668823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:45.727239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aff44-671a-4f20-9bb1-2fbdd441d5d3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:45.730877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:45.731526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:45.731797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:45.732075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:45.741450 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:45.742883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:45.752547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 988/1938] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:46.304919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:46.306792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:46.320548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90171a-6ffa-4db7-ac05-baa7723bc134 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:46.325192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:46.326075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:46.326373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:46.326625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] PATH_INFO: `/v3/projects/f42d65687a174308a488fa60bf1c6d64` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:46.327860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42d65687a174308a488fa60bf1c6d64)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:46.330007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:46.377412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42d65687a174308a488fa60bf1c6d64', 'request_id': 'req-217d4048-57d8-4e9c-9c43-bf5b2881df49'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.383679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42d65687a174308a488fa60bf1c6d64', 'request_id': 'req-217d4048-57d8-4e9c-9c43-bf5b2881df49'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.391872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42d65687a174308a488fa60bf1c6d64', 'request_id': 'req-217d4048-57d8-4e9c-9c43-bf5b2881df49'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.402696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] The token cache is being invalidate because project f42d65687a174308a488fa60bf1c6d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:46.402985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.403629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.405244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 951/1939] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:46 2022] DELETE /identity/v3/projects/f42d65687a174308a488fa60bf1c6d64 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:46.410043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:46.419735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016253-800c-4dde-846d-0b0e7b409afb service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:46.485938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:46.486816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:46.487103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:46.487380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] PATH_INFO: `/v3/users/b69be48d0d0b4902bae18265ce1d4ae5` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:46.489393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69be48d0d0b4902bae18265ce1d4ae5)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:46.492287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:46.542307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69be48d0d0b4902bae18265ce1d4ae5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.554024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69be48d0d0b4902bae18265ce1d4ae5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.554701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69be48d0d0b4902bae18265ce1d4ae5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.555299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69be48d0d0b4902bae18265ce1d4ae5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.562109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2c8493ea-2fa0-41fc-84de-f1c0d5445284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69be48d0d0b4902bae18265ce1d4ae5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.576318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 989/1940] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:46 2022] DELETE /identity/v3/users/b69be48d0d0b4902bae18265ce1d4ae5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:46.853030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:46.854557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:46.868143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4048-57d8-4e9c-9c43-bf5b2881df49 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:46.927247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:46.928028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:46.928347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:46.928629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] PATH_INFO: `/v3/projects/40d3411ca6e6421990a092f18143767a` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:46.929821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d3411ca6e6421990a092f18143767a)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:46.931734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:46.977354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d3411ca6e6421990a092f18143767a', 'request_id': 'req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.984001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d3411ca6e6421990a092f18143767a', 'request_id': 'req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:46.992199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d3411ca6e6421990a092f18143767a', 'request_id': 'req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:47.003111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] The token cache is being invalidate because project 40d3411ca6e6421990a092f18143767a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:47.003399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:47.004063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8f51f169-7d75-4a86-87f4-7e3ee19d54a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:47.005933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 952/1941] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:46 2022] DELETE /identity/v3/projects/40d3411ca6e6421990a092f18143767a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:47.010879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.011172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.011451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.031016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:47.032559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:47.092608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 990/1942] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:47.102159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.102477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.102759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.123149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:47.125110 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:47.133828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 953/1943] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:47.137960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.149080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049a249-fc3e-4410-aac4-8a40e73be67b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.155350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.156753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.157034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.157305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.158047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:47.158352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.159966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.172885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 991/1944] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:47.177360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.186375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd7b6-8760-4a28-b57c-a195c0cda295 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.191385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.192049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.192305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.192600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.193294 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.195198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.225800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 954/1945] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:53:47.230446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.242376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad19315-5e1d-4875-aadc-6fcdc3be538b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.301168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.301863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.302151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.302446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.303147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.305226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.327093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 992/1946] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:47.332005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.342343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c89b8-0ad2-4382-b992-7c82bf311e18 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.347780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.348479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.348757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.349015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.349747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.351487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.365634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 955/1947] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:47.367048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.377060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d547-67cc-43e8-a937-66ed4df471d5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.381741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.382404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.382685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.383030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.383760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.385376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547c3dd-a333-484c-9e87-8fee219a9256 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.397126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 993/1948] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:47.401047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.411065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd50c38-ca0a-4b20-b4a8-a763c9742d06 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.416021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.416947 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.417223 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.417490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] PATH_INFO: `/v3/projects/33de68b9c6de44bdb35d6b5a781fc454/users/4c377d9c7f2a484bb59d18893b6f225e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.432420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33de68b9c6de44bdb35d6b5a781fc454, user_id=4c377d9c7f2a484bb59d18893b6f225e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.434803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.456440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 956/1949] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:47 2022] PUT /identity/v3/projects/33de68b9c6de44bdb35d6b5a781fc454/users/4c377d9c7f2a484bb59d18893b6f225e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:47.467384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.468580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.469031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.500437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] MFA Rules not processed for user `4c377d9c7f2a484bb59d18893b6f225e`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:47.501681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:47.569932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 994/1950] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:47.574435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.634431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2aa6-d9a8-4ac9-acf4-2893996d43a2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.639215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.640042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.640343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.640719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.641429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.643347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.669977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 957/1951] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:47.674479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.685068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177fdc0-4a02-40da-baf2-1b5729fba1fb None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.745647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.746434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.746711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.746979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.747692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.750070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.770881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 995/1952] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:47.774996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.785397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343cc6-1a82-4476-9857-bc4a07efa1af admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.790234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.790970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.791320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.791657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.792504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.794184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.807342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 958/1953] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:47.810301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.820936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeae3bc-bb44-4bdf-a524-5b2e8e4389bd admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.825620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.826593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.826943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.827300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] PATH_INFO: `/v3/projects/dcb56534f97948bca127c0c90086b96b/users/a360e64eb6a34acaa611feb68ed14318/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.844088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb56534f97948bca127c0c90086b96b, user_id=a360e64eb6a34acaa611feb68ed14318, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.846860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.864566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 996/1954] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:47 2022] PUT /identity/v3/projects/dcb56534f97948bca127c0c90086b96b/users/a360e64eb6a34acaa611feb68ed14318/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:47.868709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.878890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10731f0-9036-4554-8a97-33097258e062 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.934484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.935222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.935404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.935719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.936459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.938274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e3ce-6008-4f7c-9dd1-27f5f4e008b1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.947872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 959/1955] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:53:47.952402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:47.963705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade30ee-0996-4264-b8b7-876d4e70b4ca admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:47.969714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:47.970600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.970880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.971154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] PATH_INFO: `/v3/projects/dcb56534f97948bca127c0c90086b96b/users/a360e64eb6a34acaa611feb68ed14318/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:47.973366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb56534f97948bca127c0c90086b96b, user_id=a360e64eb6a34acaa611feb68ed14318, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:47.975988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:47.986663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 997/1956] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:47 2022] PUT /identity/v3/projects/dcb56534f97948bca127c0c90086b96b/users/a360e64eb6a34acaa611feb68ed14318/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:47.997942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:47.998272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:47.998551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:48.029072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] MFA Rules not processed for user `a360e64eb6a34acaa611feb68ed14318`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:48.030279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:48.113795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 960/1957] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:53:48.125052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:48.126794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:48.192278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41131b20-abf2-4959-bd65-d19c30aec1a1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:48.195993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:48.196676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:48.196956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:48.197240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:48.277799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:48.280048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582568f-4091-4296-9ada-e78ca7440da5 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:48.289785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 998/1958] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:57.122624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:57.189175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cc865-bad0-496e-84ec-eee6fc02c7f0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:57.194667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:57.195666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:57.195948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:57.196221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] PATH_INFO: `/v3/users/4c377d9c7f2a484bb59d18893b6f225e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:57.197576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c377d9c7f2a484bb59d18893b6f225e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:57.199508 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:57.253489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c377d9c7f2a484bb59d18893b6f225e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.264809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c377d9c7f2a484bb59d18893b6f225e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.271707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c377d9c7f2a484bb59d18893b6f225e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.272525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c377d9c7f2a484bb59d18893b6f225e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.281864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c377d9c7f2a484bb59d18893b6f225e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.289061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 961/1959] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:57 2022] DELETE /identity/v3/users/4c377d9c7f2a484bb59d18893b6f225e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:57.294857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:57.295382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:57.295382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:57.313333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:57.314574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:57.374903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 999/1960] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:57.384976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:57.386537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:57.447669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a731-5109-4882-8ab5-9ea34a6be51d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:57.452161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:57.452840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:57.453115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:57.453369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:57.471481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:57.473696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:57.483717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 962/1961] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:53:57.838110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:57.839751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:57.852240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fd16c-a790-44d0-b605-c3844038c3c7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:57.858926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:57.860231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:57.860609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:57.860891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] PATH_INFO: `/v3/projects/33de68b9c6de44bdb35d6b5a781fc454` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:57.862549 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33de68b9c6de44bdb35d6b5a781fc454)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:57.864961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:57.922942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33de68b9c6de44bdb35d6b5a781fc454', 'request_id': 'req-a3836f69-62df-48ed-9e56-daf64e6b9c94'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.930070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33de68b9c6de44bdb35d6b5a781fc454', 'request_id': 'req-a3836f69-62df-48ed-9e56-daf64e6b9c94'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.941894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33de68b9c6de44bdb35d6b5a781fc454', 'request_id': 'req-a3836f69-62df-48ed-9e56-daf64e6b9c94'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.954819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] The token cache is being invalidate because project 33de68b9c6de44bdb35d6b5a781fc454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:57.955121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.955851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:57.958085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1000/1962] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:57 2022] DELETE /identity/v3/projects/33de68b9c6de44bdb35d6b5a781fc454 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:57.964749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:57.979203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7c58c-989a-449b-886c-612d60fc248d service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:58.091787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:58.092814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.093333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.093673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] PATH_INFO: `/v3/users/a360e64eb6a34acaa611feb68ed14318` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.095500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a360e64eb6a34acaa611feb68ed14318)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:58.098043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:58.167259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a360e64eb6a34acaa611feb68ed14318'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.182298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a360e64eb6a34acaa611feb68ed14318'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.190206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a360e64eb6a34acaa611feb68ed14318'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.190989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a360e64eb6a34acaa611feb68ed14318'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.201342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbf30249-9e24-451a-974d-6a203ec1d0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a360e64eb6a34acaa611feb68ed14318'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.210741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 963/1963] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:53:57 2022] DELETE /identity/v3/users/a360e64eb6a34acaa611feb68ed14318 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:58.626651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:58.628319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:58.645126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836f69-62df-48ed-9e56-daf64e6b9c94 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:58.711870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:58.713118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.713407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.713685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] PATH_INFO: `/v3/projects/dcb56534f97948bca127c0c90086b96b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.715038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb56534f97948bca127c0c90086b96b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:58.717016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:58.763645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb56534f97948bca127c0c90086b96b', 'request_id': 'req-933cbb07-a97c-4a6c-a1f3-5218359826df'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.769848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb56534f97948bca127c0c90086b96b', 'request_id': 'req-933cbb07-a97c-4a6c-a1f3-5218359826df'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.778202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb56534f97948bca127c0c90086b96b', 'request_id': 'req-933cbb07-a97c-4a6c-a1f3-5218359826df'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.790064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] The token cache is being invalidate because project dcb56534f97948bca127c0c90086b96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:53:58.790356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.791111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-933cbb07-a97c-4a6c-a1f3-5218359826df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:53:58.792981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1001/1964] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:53:58 2022] DELETE /identity/v3/projects/dcb56534f97948bca127c0c90086b96b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:53:58.798383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.798750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.799098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.817561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:58.818892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:58.886689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 964/1965] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:58.894472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.894794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.895133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.915193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:58.916471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:58.925913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1002/1966] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:58.930393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:58.940388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4144-cb23-4f4f-8123-3137b1d55a29 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:58.946560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:58.947224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.947507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.947783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.948547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:53:58.948857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:58.950468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:58.962854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 965/1967] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:53:58.967361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:58.977135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd1864-b6b4-48d7-b73d-707ab2d409e0 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:58.983188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:58.983845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:58.984130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:58.984448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:58.985148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:58.987059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.016073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1003/1968] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:53:58 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:59.020872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:59.035070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf0467-9655-47b2-88ca-034c08b2e098 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:59.093908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:59.094588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.094841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.095117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.095844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:59.098048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.120666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 966/1969] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:53:59 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:53:59.125067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:59.136287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4242-6547-4478-bb81-b5ab90c8844a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:59.141372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:59.142061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.142350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.142628 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.143373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:59.145067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.156144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1004/1970] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:59.160663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:59.170466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dd661-01ba-437f-ad6c-a8761b0a8ad7 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:59.175163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:59.175832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.176115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.176470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.177211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:59.178824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e32c-366c-43d1-9aca-90d6a411882f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.193788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 967/1971] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:53:59.194954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:59.204750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93aeff-4f12-4b4a-8716-914550555c5d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:59.209600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:59.210483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.210764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.211037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] PATH_INFO: `/v3/projects/a7981466c1984f26a302a2498dfb7d36/users/aa4e72d435db458b8490965138dad88e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.230518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7981466c1984f26a302a2498dfb7d36, user_id=aa4e72d435db458b8490965138dad88e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:59.234247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.250605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1005/1972] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:53:59 2022] PUT /identity/v3/projects/a7981466c1984f26a302a2498dfb7d36/users/aa4e72d435db458b8490965138dad88e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:53:59.261056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.261338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.261649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.289537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] MFA Rules not processed for user `aa4e72d435db458b8490965138dad88e`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:53:59.290760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:59.354313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 968/1973] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:53:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:53:59.366724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:53:59.368460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:53:59.427772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c33b50-ff1e-40d7-88ce-ca81df24ea0a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:53:59.431402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:53:59.432056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:53:59.432371 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:53:59.432676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:53:59.444439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:53:59.445867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5b344-5c18-4955-8d64-ac1dc064e0f0 service glance] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:53:59.454509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1006/1974] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:53:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:08.809559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:08.811243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:08.932099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab8d2-e42a-4e8f-9f0a-105176d244c0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:08.937602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:08.938680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:08.938994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:08.939216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] PATH_INFO: `/v3/users/486c247a63e04e159b7d911e4239ed35` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:08.940488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] RBAC: Authorizing `identity:delete_user(user_id=486c247a63e04e159b7d911e4239ed35)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:08.942430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:09.009415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486c247a63e04e159b7d911e4239ed35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.023840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486c247a63e04e159b7d911e4239ed35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.036453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486c247a63e04e159b7d911e4239ed35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.038195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486c247a63e04e159b7d911e4239ed35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.048231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486c247a63e04e159b7d911e4239ed35'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.055465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 969/1975] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:08 2022] DELETE /identity/v3/users/486c247a63e04e159b7d911e4239ed35 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:09.060423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:09.060953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:09.060953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:09.091524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:09.093284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:09.162759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1007/1976] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:09.174373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:09.175925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:09.246913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf582-598b-4c38-8d69-6d87db56c68c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:09.250662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:09.253777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:09.255071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:09.255428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:09.276518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:09.276518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e0dd0-e10b-44fc-a583-7e30f307e9a9 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:09.292956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 970/1977] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:09.773076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:09.774659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:09.786881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e464d-85aa-4437-84ce-78eb5f50c293 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:09.791564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:09.792556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:09.792838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:09.793110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] PATH_INFO: `/v3/projects/60176266845d46219a1493294f118e94` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:09.794372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60176266845d46219a1493294f118e94)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:09.796339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:09.852829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60176266845d46219a1493294f118e94', 'request_id': 'req-081a32ad-daef-4003-a568-d9a4fc807520'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.859898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60176266845d46219a1493294f118e94', 'request_id': 'req-081a32ad-daef-4003-a568-d9a4fc807520'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.869607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60176266845d46219a1493294f118e94', 'request_id': 'req-081a32ad-daef-4003-a568-d9a4fc807520'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.882593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] The token cache is being invalidate because project 60176266845d46219a1493294f118e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:09.882883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.883589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-081a32ad-daef-4003-a568-d9a4fc807520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:09.885463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1008/1978] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:09 2022] DELETE /identity/v3/projects/60176266845d46219a1493294f118e94 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:09.890427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:09.891318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:09.891823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:09.911154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:09.912504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:09.981050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 971/1979] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:09.982803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:09.983090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:09.983369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.005366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:10.006976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:10.022519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1009/1980] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:10.026930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.043532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f30942-57bc-4ee3-9215-5d7f1cc1c793 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.049994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.050625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.050910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.051346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.052084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:10.052432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.054302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.068479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 972/1981] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:10.072547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.093252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f377bf4-f306-4d60-8fb8-bfbfbc402975 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.098104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.098768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.100684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.100964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.101677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.103596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.131663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1010/1982] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:10.135954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.152204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770cd9-e8f9-495e-8bf3-d89f59057a59 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.212196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.212926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.213250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.213466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.214550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.216845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.240015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 973/1983] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:10 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:10.244911 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.265272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e127af8-6a65-4492-9bda-a41ab4235f0c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.272857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.273677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.273898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.274207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.275184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.277711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.298831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1011/1984] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:10.302823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.331517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23c53-2011-49ed-a389-a9d795d82ef0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.344598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.346408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.348798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.349357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.351256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.353347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe5eb6-c656-40bf-92eb-6928ab6e6780 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.367017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 974/1985] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:10.371194 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.381434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fcb2-a324-43af-94c9-0b1f10d38c70 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.386068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.387032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.387337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.387619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] PATH_INFO: `/v3/projects/928d38a4abeb4bd08b5f68ab76322791/users/943cb09d363942269493db78b38585cd/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.402254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] RBAC: Authorizing `identity:create_grant(project_id=928d38a4abeb4bd08b5f68ab76322791, user_id=943cb09d363942269493db78b38585cd, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.404934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.425915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1012/1986] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:10 2022] PUT /identity/v3/projects/928d38a4abeb4bd08b5f68ab76322791/users/943cb09d363942269493db78b38585cd/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:54:10.437685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.438003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.438277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.472027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] MFA Rules not processed for user `943cb09d363942269493db78b38585cd`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:10.473358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:10.542472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 975/1987] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:10.554185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.555842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:10.617558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1535c7-ee7d-4a25-a0bd-f7fe0d6364da admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.622169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.622875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.623154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.623447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.633757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.635202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30dbc9a-fcc1-45be-8306-cdb98b63be2a service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.645320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1013/1988] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:10.677749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.740198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a49d72-a256-42a3-9c3e-a23b73ab1552 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:10.745578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:10.746449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.746735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.747006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] PATH_INFO: `/v3/users/943cb09d363942269493db78b38585cd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.748267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=943cb09d363942269493db78b38585cd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:10.750208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:10.807430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943cb09d363942269493db78b38585cd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:10.820581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943cb09d363942269493db78b38585cd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:10.827411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943cb09d363942269493db78b38585cd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:10.828138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943cb09d363942269493db78b38585cd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:10.837817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943cb09d363942269493db78b38585cd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:10.843992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 976/1989] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:10 2022] DELETE /identity/v3/users/943cb09d363942269493db78b38585cd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:10.848684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:10.848976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:10.849258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:10.867759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:10.868660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:10.942260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1014/1990] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:10.952031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:10.953542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:11.009745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394097f-182a-45a2-8bf6-297d121d5109 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.013815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.014459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.014742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.015019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.024427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.025864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b62a71-895e-4ca6-b1f4-c42ad690647a service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.037661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 977/1991] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:11.319423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.321751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:11.335283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef1b58-e041-4534-aaef-44bb38dfd256 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.343028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.344179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.344916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.345199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] PATH_INFO: `/v3/projects/928d38a4abeb4bd08b5f68ab76322791` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.346617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928d38a4abeb4bd08b5f68ab76322791)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.349514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.405218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928d38a4abeb4bd08b5f68ab76322791', 'request_id': 'req-795efa45-1a75-4d73-81c0-9b069bda8727'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:11.411460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928d38a4abeb4bd08b5f68ab76322791', 'request_id': 'req-795efa45-1a75-4d73-81c0-9b069bda8727'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:11.418959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928d38a4abeb4bd08b5f68ab76322791', 'request_id': 'req-795efa45-1a75-4d73-81c0-9b069bda8727'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:11.429347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] The token cache is being invalidate because project 928d38a4abeb4bd08b5f68ab76322791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:11.429630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:11.430253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-795efa45-1a75-4d73-81c0-9b069bda8727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:11.432052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1015/1992] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:11 2022] DELETE /identity/v3/projects/928d38a4abeb4bd08b5f68ab76322791 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:11.437597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.438221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.438782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.460208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:11.461480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:11.526704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 978/1993] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:11.530179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.530278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.530490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.550235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:11.551439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:11.562930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1016/1994] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:11.567175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.577022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cffbe-a948-4f12-b3a6-99b3eb8190ac None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.581498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.582157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.582437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.582711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.583441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:11.583781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.585408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.597902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 979/1995] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:11.602019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.611126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97583-93d1-45c6-a525-0db6d369305d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.616357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.617068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.617293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.617564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.618269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.620177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.653954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1017/1996] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:11 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:11.658674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.669741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27e49-2b52-40f9-b0c8-a9856a8eec19 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.723097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.723814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.724093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.724575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.725365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.727309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.748563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 980/1997] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:11 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:54:11.752903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.763481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02951820-0971-4bca-969e-14b1cbefbbc1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.768359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.769012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.769293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.769569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.770304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.772133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.783044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1018/1998] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:11.787586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.797485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25973d-8de5-4a1c-b005-e5be485af4a1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.802227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.802924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.803212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.803482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.804217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.805844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a0e03-d1a6-4724-a98a-5f6e124a2387 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.815865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 981/1999] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:11.820197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:11.830288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6a107-7f28-423b-9789-1595175a47ce admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:11.835372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:11.836609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.836897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.837158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] PATH_INFO: `/v3/projects/bb0e8c31f64c4da1ae4ad2ca1f6c98be/users/068de2d93f4f4d10ac7180fc1625999d/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.864198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0e8c31f64c4da1ae4ad2ca1f6c98be, user_id=068de2d93f4f4d10ac7180fc1625999d, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:11.867145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:11.883116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1019/2000] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:11 2022] PUT /identity/v3/projects/bb0e8c31f64c4da1ae4ad2ca1f6c98be/users/068de2d93f4f4d10ac7180fc1625999d/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:11.894252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0250-3a51-416b-aad0-4bba2ea3f42e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:11.894564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0250-3a51-416b-aad0-4bba2ea3f42e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:11.894846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0250-3a51-416b-aad0-4bba2ea3f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:11.921702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c1cc0250-3a51-416b-aad0-4bba2ea3f42e None None] MFA Rules not processed for user `068de2d93f4f4d10ac7180fc1625999d`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:11.922913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c1cc0250-3a51-416b-aad0-4bba2ea3f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:11.994727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 982/2001] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:12.008332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:12.010046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:12.084634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8c6e2-32cc-4bc7-afff-5498c12ef042 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:12.088576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:12.089469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:12.090382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:12.090663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:12.100208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:12.101698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:12.115615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1020/2002] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:13.493677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:13.494206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:13.494206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:13.512160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] MFA Rules not processed for user `068de2d93f4f4d10ac7180fc1625999d`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:13.513395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:13.520201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 983/2003] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:21.049588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.051913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:21.128380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7e01d-f262-4412-adcb-e940af2b7703 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.135971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.137377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.137794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.138205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] PATH_INFO: `/v3/users/2e8eb473129a4bda9eb87010e961d4ab` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.140239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8eb473129a4bda9eb87010e961d4ab)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.142785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.197029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8eb473129a4bda9eb87010e961d4ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.210111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8eb473129a4bda9eb87010e961d4ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.210941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8eb473129a4bda9eb87010e961d4ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.211642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8eb473129a4bda9eb87010e961d4ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.218517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bbfd9779-5274-484f-8e58-95ebff835150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8eb473129a4bda9eb87010e961d4ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.233961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1021/2004] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:21 2022] DELETE /identity/v3/users/2e8eb473129a4bda9eb87010e961d4ab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:21.416988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.430275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a641874-7e60-4fb5-b3c7-197446e39ea6 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.489098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.490084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.490838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.490838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] PATH_INFO: `/v3/projects/c1fd9534b0c84b8f85a8793caef79d3f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.491925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fd9534b0c84b8f85a8793caef79d3f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.493881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.544913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fd9534b0c84b8f85a8793caef79d3f', 'request_id': 'req-86b6e229-56cf-4693-8c78-289698d7260e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.551843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fd9534b0c84b8f85a8793caef79d3f', 'request_id': 'req-86b6e229-56cf-4693-8c78-289698d7260e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.561329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fd9534b0c84b8f85a8793caef79d3f', 'request_id': 'req-86b6e229-56cf-4693-8c78-289698d7260e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.572739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] The token cache is being invalidate because project c1fd9534b0c84b8f85a8793caef79d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:21.573028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.573741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-86b6e229-56cf-4693-8c78-289698d7260e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:21.575705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 984/2005] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:21 2022] DELETE /identity/v3/projects/c1fd9534b0c84b8f85a8793caef79d3f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:21.580692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.581059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.581342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.601582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:21.602234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:21.669076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1022/2006] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:21.677227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.677657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.678469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.697852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:21.699036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:21.707986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 985/2007] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:21.712537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.723627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466d0f9-49bf-4295-b945-432ed3e053da None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.729678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.730506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.730902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.731235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.732046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:21.732565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.734359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.747978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1023/2008] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:21.752525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.762201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd126a-7348-4c72-9c31-74197930f24e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.767132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.767796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.768075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.768374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.769062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.770965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.796643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 986/2009] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:21 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:21.800923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.811277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad334c-e863-4135-87fd-28cca3faa696 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.885026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.885996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.886321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.886677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.887776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.890825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.915756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1024/2010] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:21 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:21.920035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.930241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7245be5-93b5-4e69-859a-2ec2be7fdab3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.935226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.935874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.936151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.936454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.937175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.938773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.950382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 987/2011] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:21.955819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:21.966621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581f89b-261e-4cc4-802d-4cc0b19dfa11 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:21.972349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:21.973003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:21.973307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:21.973583 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:21.974628 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:21.976271 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46b904-6259-480c-8184-cdee30b935d5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:21.986110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1025/2012] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:54:21.990968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:22.000629 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bff40-d415-4b36-aca9-5a4f8bc0fb08 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:22.005935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:22.006815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.007094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.007365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] PATH_INFO: `/v3/projects/71e05da90f5e475d8d9392ddf9857264/users/269390bddf144f3d982c0a8c089ad49b/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.022544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e05da90f5e475d8d9392ddf9857264, user_id=269390bddf144f3d982c0a8c089ad49b, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:22.025242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:22.041000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 988/2013] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:21 2022] PUT /identity/v3/projects/71e05da90f5e475d8d9392ddf9857264/users/269390bddf144f3d982c0a8c089ad49b/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:22.051359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.051838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.051973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.081011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] MFA Rules not processed for user `269390bddf144f3d982c0a8c089ad49b`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:22.082212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:22.147747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1026/2014] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:22.159073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:22.162060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:22.258074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f61c8-e22b-4b48-b22e-860936228d5e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:22.263826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:22.264510 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.264782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.265051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.275395 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:22.278060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44023-f2a2-4363-811f-e9209cde534b service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:22.290856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 989/2015] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:22.522735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:22.594031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4198c-4694-442a-9626-5e5d74538142 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:22.600330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:22.601208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.601485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.601755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] PATH_INFO: `/v3/users/269390bddf144f3d982c0a8c089ad49b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.603171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=269390bddf144f3d982c0a8c089ad49b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:22.605144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:22.663254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269390bddf144f3d982c0a8c089ad49b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:22.675236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269390bddf144f3d982c0a8c089ad49b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:22.675975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269390bddf144f3d982c0a8c089ad49b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:22.676611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269390bddf144f3d982c0a8c089ad49b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:22.684235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269390bddf144f3d982c0a8c089ad49b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:22.701327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1027/2016] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:22 2022] DELETE /identity/v3/users/269390bddf144f3d982c0a8c089ad49b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:22.708235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.709174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.709460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.727865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:22.729124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:22.790970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 990/2017] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:22.800461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:22.802032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:22.866412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdd543-c8c5-44b4-856b-05741a9af0dc admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:22.870032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:22.870680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:22.870961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:22.871237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:22.880486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:22.882378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1217c76-8490-49d2-9aa6-dfa3a13d907d service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:22.892804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1028/2018] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:23.168505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.177889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:23.190126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d928-db3e-45f7-abba-21778b11cb0e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.195079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.195948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.196233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.196584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] PATH_INFO: `/v3/projects/71e05da90f5e475d8d9392ddf9857264` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.197805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e05da90f5e475d8d9392ddf9857264)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.199666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.246720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e05da90f5e475d8d9392ddf9857264', 'request_id': 'req-2339cc85-0758-43ea-a65c-6c3549f154e8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:23.252763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e05da90f5e475d8d9392ddf9857264', 'request_id': 'req-2339cc85-0758-43ea-a65c-6c3549f154e8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:23.261698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e05da90f5e475d8d9392ddf9857264', 'request_id': 'req-2339cc85-0758-43ea-a65c-6c3549f154e8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:23.273485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] The token cache is being invalidate because project 71e05da90f5e475d8d9392ddf9857264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:23.273776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:23.274482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-2339cc85-0758-43ea-a65c-6c3549f154e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:23.276158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 991/2019] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:23 2022] DELETE /identity/v3/projects/71e05da90f5e475d8d9392ddf9857264 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:23.281365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.281936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.281936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.301714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:23.303119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:23.372403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1029/2020] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:23.381146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.381146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.381414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.401553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:23.402771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:23.415321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 992/2021] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:23.419791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.430497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882f2bb-e69f-492b-a6bb-8a1a7a31deb8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.436274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.436955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.437242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.437522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.438262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:23.438568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.440190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.452956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1030/2022] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:23.457423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.466915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ae7de-dbb3-418a-bfc0-870c73842685 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.472412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.473078 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.473365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.473637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.474379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.476330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.508674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 993/2023] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:23 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:23.512825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.522849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb68af-84a8-41e6-a870-6bfb4d8e40d3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.578547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.579226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.579510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.579785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.580504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.582449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.607171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1031/2024] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:23 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:23.608435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.619979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ff16-c046-472b-91a3-6eee5d55e212 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.625941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.626748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.627128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.627498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.628413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.630366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.642263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 994/2025] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:23.646797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.657321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3c32-eb67-4446-bc69-0482bd54a4cb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.662892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.663618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.663897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.664167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.664952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.666543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e345d31-edf6-4c91-aae4-54804f3c2bcc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.676811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1032/2026] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:23.681040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.690913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a100f-0ee7-4e17-afde-b61485beda2c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.695685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.696605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.696891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.697171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] PATH_INFO: `/v3/projects/ffe75b1e61df47b2abd29ea2d5fe2f60/users/8f7c0aeb8e654517b1d185d4ccb36c4a/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.712433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe75b1e61df47b2abd29ea2d5fe2f60, user_id=8f7c0aeb8e654517b1d185d4ccb36c4a, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.715087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.729670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 995/2027] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:23 2022] PUT /identity/v3/projects/ffe75b1e61df47b2abd29ea2d5fe2f60/users/8f7c0aeb8e654517b1d185d4ccb36c4a/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:23.734919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd0fd1-fe48-46d4-9a40-a186146705e8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.735308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd0fd1-fe48-46d4-9a40-a186146705e8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.735654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd0fd1-fe48-46d4-9a40-a186146705e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.754635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-efdd0fd1-fe48-46d4-9a40-a186146705e8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:23.755922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-efdd0fd1-fe48-46d4-9a40-a186146705e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:23.815789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1033/2028] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:23.826043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:23.827796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:23.884345 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eab95-806e-49ac-b8d6-b849982c7546 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:23.888560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:23.889218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:23.889500 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:23.889777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:23.899087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:23.900543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:23.910197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 996/2029] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:24.564224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.564766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.565603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.596622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] MFA Rules not processed for user `8f7c0aeb8e654517b1d185d4ccb36c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:24.597831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:24.667053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1034/2030] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:24.672183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:24.682468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b68fd-0689-4314-8d3c-659b4bea9327 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:24.688056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:24.688744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.689031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.689306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.690037 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:24.691937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:24.719968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 997/2031] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:24 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:24.724078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:24.734935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e765e-2ed3-445a-88af-68feba078e0a None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:24.791485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:24.792174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.792494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.792773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.793480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:24.795395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:24.817263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1035/2032] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:24 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:24.821541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:24.832402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe183d19-aa3e-4154-abb7-23c54938ea38 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:24.838134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:24.839012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.839295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.839568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.840341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:24.841956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:24.855255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 998/2033] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:24.859479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:24.870143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b21e8-51b1-4124-8270-444cd2c3b421 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:24.874902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:24.875798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.876079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.876394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] PATH_INFO: `/v3/projects/055e9f33591b4ce8ab184c3116e9e1db/users/3453bdab78434ba2a8b1eab947683819/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.891200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] RBAC: Authorizing `identity:create_grant(project_id=055e9f33591b4ce8ab184c3116e9e1db, user_id=3453bdab78434ba2a8b1eab947683819, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:24.894016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:24.909190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1036/2034] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:24 2022] PUT /identity/v3/projects/055e9f33591b4ce8ab184c3116e9e1db/users/3453bdab78434ba2a8b1eab947683819/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:24.913404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:24.923404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab03ee-4a06-4113-97b4-cd2ad962358b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:24.986968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:24.988052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:24.988773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:24.989303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:24.990558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:24.992387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abcd68-ab0a-4d78-b3f8-8ea31e9577b8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:25.003617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 999/2035] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:25.008656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:25.018879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aae2-447f-40fd-8830-2e7e2e5d335e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:25.023900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:25.024828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:25.025079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:25.025350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] PATH_INFO: `/v3/projects/055e9f33591b4ce8ab184c3116e9e1db/users/3453bdab78434ba2a8b1eab947683819/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:25.027318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055e9f33591b4ce8ab184c3116e9e1db, user_id=3453bdab78434ba2a8b1eab947683819, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:25.029976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:25.041789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1037/2036] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:25 2022] PUT /identity/v3/projects/055e9f33591b4ce8ab184c3116e9e1db/users/3453bdab78434ba2a8b1eab947683819/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:25.724083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c311df-058b-4773-acb8-a746fccffaed None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:25.724652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c311df-058b-4773-acb8-a746fccffaed None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:25.724851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c311df-058b-4773-acb8-a746fccffaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:25.751439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-11c311df-058b-4773-acb8-a746fccffaed None None] MFA Rules not processed for user `3453bdab78434ba2a8b1eab947683819`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:25.752729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-11c311df-058b-4773-acb8-a746fccffaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:25.825238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1000/2037] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:27.069636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:27.138308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f0839-73ab-4ff7-ac2f-35abbedaf904 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:27.143782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:27.144719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:27.145022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:27.145294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] PATH_INFO: `/v3/users/8f7c0aeb8e654517b1d185d4ccb36c4a` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:27.146844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7c0aeb8e654517b1d185d4ccb36c4a)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:27.148890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:27.198018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7c0aeb8e654517b1d185d4ccb36c4a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.210540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7c0aeb8e654517b1d185d4ccb36c4a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.211323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7c0aeb8e654517b1d185d4ccb36c4a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.211931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7c0aeb8e654517b1d185d4ccb36c4a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.218605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7c0aeb8e654517b1d185d4ccb36c4a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.232942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1038/2038] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:27 2022] DELETE /identity/v3/users/8f7c0aeb8e654517b1d185d4ccb36c4a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:27.407695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c311df-058b-4773-acb8-a746fccffaed None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:27.419460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c311df-058b-4773-acb8-a746fccffaed None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:27.474998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:27.475857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:27.476132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:27.476429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] PATH_INFO: `/v3/projects/ffe75b1e61df47b2abd29ea2d5fe2f60` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:27.477611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe75b1e61df47b2abd29ea2d5fe2f60)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:27.479521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:27.525176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe75b1e61df47b2abd29ea2d5fe2f60', 'request_id': 'req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.531098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe75b1e61df47b2abd29ea2d5fe2f60', 'request_id': 'req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.540560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe75b1e61df47b2abd29ea2d5fe2f60', 'request_id': 'req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.556679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] The token cache is being invalidate because project ffe75b1e61df47b2abd29ea2d5fe2f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:27.557069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.558181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.560361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1001/2039] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:27 2022] DELETE /identity/v3/projects/ffe75b1e61df47b2abd29ea2d5fe2f60 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:54:27.564544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:27.566109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:27.579193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfcf6f-e532-45f6-98a1-dc94075d53ba admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:27.648962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:27.649835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:27.650113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:27.650383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] PATH_INFO: `/v3/users/3453bdab78434ba2a8b1eab947683819` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:27.651582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3453bdab78434ba2a8b1eab947683819)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:27.653534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:27.703200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3453bdab78434ba2a8b1eab947683819'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.716672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3453bdab78434ba2a8b1eab947683819'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.717357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3453bdab78434ba2a8b1eab947683819'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.718048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3453bdab78434ba2a8b1eab947683819'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.725239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0570d555-151c-409b-bf1d-b60a854a76ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3453bdab78434ba2a8b1eab947683819'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:27.740482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1039/2040] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:27 2022] DELETE /identity/v3/users/3453bdab78434ba2a8b1eab947683819 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:27.921015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:27.922515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:27.936231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc3a7c-f908-4d0b-ac5f-4669cedc155a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:27.998390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:27.999262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:27.999550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:27.999829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] PATH_INFO: `/v3/projects/055e9f33591b4ce8ab184c3116e9e1db` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.001050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=055e9f33591b4ce8ab184c3116e9e1db)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.002964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.050566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055e9f33591b4ce8ab184c3116e9e1db', 'request_id': 'req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:28.058064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055e9f33591b4ce8ab184c3116e9e1db', 'request_id': 'req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:28.067410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055e9f33591b4ce8ab184c3116e9e1db', 'request_id': 'req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:28.079702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] The token cache is being invalidate because project 055e9f33591b4ce8ab184c3116e9e1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:28.079987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:28.080726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-64a8cca8-ab40-420e-8eb4-d8b106e7c15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:28.082311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1002/2041] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:27 2022] DELETE /identity/v3/projects/055e9f33591b4ce8ab184c3116e9e1db => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:28.087661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.088163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.088328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.105845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:28.107051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:28.165922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1040/2042] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:54:28.174421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a93-079d-461d-91e0-39729f42b970 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.174870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a93-079d-461d-91e0-39729f42b970 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.175196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a93-079d-461d-91e0-39729f42b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.195434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-49619a93-079d-461d-91e0-39729f42b970 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:28.196687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-49619a93-079d-461d-91e0-39729f42b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:28.206523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1003/2043] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:28.212174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.223166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10646-c44f-4bbb-a7fb-3d828270f795 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.229701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.230486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.230817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.231124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.232164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:28.232681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.234436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.247493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1041/2044] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:28.251879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49619a93-079d-461d-91e0-39729f42b970 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.262025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49619a93-079d-461d-91e0-39729f42b970 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.267230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.267889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.268177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.268484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.269195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.271105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.305507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1004/2045] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:28.311966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.324626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f62c0-c76c-4ac9-9cc3-41986d3005f6 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.381592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.382300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.382577 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.382848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.383533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.385476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.410787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1042/2046] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:28.414459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.429649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960db6-b322-4c44-9120-a6d684777753 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.435140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.435775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.436055 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.436357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.437202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.438858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.449692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1005/2047] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:28.453798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.465036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1ecf6-d995-4a47-8247-9d6f2b0bf084 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.469930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.470709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.470999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.471275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.472007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.473627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fce14-cf46-44de-be96-38a9b7a8ee2c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.484928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1043/2048] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:28.489676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.500092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114db4-fa36-4695-857c-0627ef4d8ed6 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.505282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.506166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.506483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.506779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] PATH_INFO: `/v3/projects/e4111340ce99406d8ed0502637a0baee/users/8b24fdd8295f46a0af75d45a5e3f51a7/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.521254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4111340ce99406d8ed0502637a0baee, user_id=8b24fdd8295f46a0af75d45a5e3f51a7, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.524171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.540827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1006/2049] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:28 2022] PUT /identity/v3/projects/e4111340ce99406d8ed0502637a0baee/users/8b24fdd8295f46a0af75d45a5e3f51a7/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:28.551656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.551945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.552265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.579400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] MFA Rules not processed for user `8b24fdd8295f46a0af75d45a5e3f51a7`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:28.580659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:28.650730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1044/2050] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:28.655386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.722505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a004ff-e2c5-4008-89f2-b9a55e6acde1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.726192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.726928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.727211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.727484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.728317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.730339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.763672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1007/2051] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:28.769106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.780567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cd91e-4cf1-4858-a143-375edb166b2c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.836340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.837262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.837665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.837937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.838777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.840800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.863766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1045/2052] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:28 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:28.869302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.880758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb1478-d577-419a-9246-1a64c9a9a843 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.887181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.887843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.888126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.888443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.889185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.890902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.902031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1008/2053] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:28.906368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.917210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00a4c2-9078-4c00-9ada-ef33dbe95d2c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:28.922513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:28.923397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:28.923676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:28.923951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] PATH_INFO: `/v3/projects/26c7796dc2054065a9275b1a58feee6c/users/995bd1bd26fe4b3c87d3018484655703/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:28.940965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c7796dc2054065a9275b1a58feee6c, user_id=995bd1bd26fe4b3c87d3018484655703, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:28.943662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:28.957726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1046/2054] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:28 2022] PUT /identity/v3/projects/26c7796dc2054065a9275b1a58feee6c/users/995bd1bd26fe4b3c87d3018484655703/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:28.962045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:28.972087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfd987-43d5-4be2-8de7-55698b5b1cea admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:29.033569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:29.034243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:29.034521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:29.034798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:29.035548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:29.037187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51155de-5c20-4805-887f-274b09c0387e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:29.048102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1009/2055] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:29.052286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:29.062970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef7b72-d683-4448-9aaa-dd9a58e1eb96 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:29.067965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:29.070502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:29.070502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:29.070502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] PATH_INFO: `/v3/projects/26c7796dc2054065a9275b1a58feee6c/users/995bd1bd26fe4b3c87d3018484655703/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:29.072520 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c7796dc2054065a9275b1a58feee6c, user_id=995bd1bd26fe4b3c87d3018484655703, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:29.075327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:29.087054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1047/2056] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:29 2022] PUT /identity/v3/projects/26c7796dc2054065a9275b1a58feee6c/users/995bd1bd26fe4b3c87d3018484655703/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:29.098482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:29.098770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:29.099051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:29.126136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] MFA Rules not processed for user `995bd1bd26fe4b3c87d3018484655703`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:29.127406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:29.202305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1010/2057] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:29.212345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:29.214663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:29.277539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e378c-c3c0-4fdf-9864-4a644d24acec admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:29.281346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:29.282001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:29.282273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:29.282546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:29.292118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:29.293640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc298b8-57ac-4987-a4ee-0b1f92ea1326 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:29.303103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1048/2058] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:29.378427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:29.388939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e85cd2-ac64-4542-a05b-60f4136383bc None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:29.392801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:29.393553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:29.393899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:29.394240 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:29.453046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:29.454477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:29.463624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1011/2059] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:37.793765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfcd4-51ce-41a8-95ca-ab11ddae8d37 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:37.794323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfcd4-51ce-41a8-95ca-ab11ddae8d37 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:37.794323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfcd4-51ce-41a8-95ca-ab11ddae8d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:37.815348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-f56cfcd4-51ce-41a8-95ca-ab11ddae8d37 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:37.816588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f56cfcd4-51ce-41a8-95ca-ab11ddae8d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:37.888678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1049/2060] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:54:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:37.934522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:37.936278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:38.006021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d068-b904-4247-b8f7-5e019aef6043 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:38.012291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:38.012935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:38.013216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:38.013488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:38.021814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:38.023538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:38.032225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1012/2061] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:54:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:54:38.599746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8317b83-dece-4e7e-9679-fba93d134f74 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:38.600230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8317b83-dece-4e7e-9679-fba93d134f74 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:38.600808 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8317b83-dece-4e7e-9679-fba93d134f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:38.619625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e8317b83-dece-4e7e-9679-fba93d134f74 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:38.620841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e8317b83-dece-4e7e-9679-fba93d134f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:38.628759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1050/2062] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 12 00:54:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:38.642131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:38.652782 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a859-b807-4e9a-be9c-f60be881fae1 service swift] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:38.658312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:38.658946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:38.659224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:38.659498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:38.667755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:38.669351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:38.677139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1013/2063] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:54:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:54:38.886030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1ef8-9e8f-4ed1-9796-d9d57fbabe87 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:38.886486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1ef8-9e8f-4ed1-9796-d9d57fbabe87 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:38.886633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1ef8-9e8f-4ed1-9796-d9d57fbabe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:38.904810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1a1c1ef8-9e8f-4ed1-9796-d9d57fbabe87 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:38.906018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1a1c1ef8-9e8f-4ed1-9796-d9d57fbabe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:38.914575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1051/2064] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:54:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:38.927013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:38.938012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d6a32-d9f7-48e8-95da-4e59647200cc service swift] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:38.942980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:38.943638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:38.943915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:38.944183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:38.952369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:38.953911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:38.962432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1014/2065] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:54:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:54:39.304871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:39.305158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:39.305473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:39.323100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:39.324287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:39.331553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1052/2066] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 12 00:54:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:39.343727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:39.356399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218201d-92de-4a70-be77-41f413eeff6e service swift] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:39.364848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:39.365743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:39.366155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:39.366498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:39.375904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:39.377541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb1178-dd70-46d9-a746-cb1a37472485 service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:39.386189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1015/2067] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:54:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:54:45.984694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:45.986880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.053031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8d2a7-63f3-4d64-95bf-352385f7ca00 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.057905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.058785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.059062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.059336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] PATH_INFO: `/v3/users/068de2d93f4f4d10ac7180fc1625999d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.060645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068de2d93f4f4d10ac7180fc1625999d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.062573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:46.110191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068de2d93f4f4d10ac7180fc1625999d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.123173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068de2d93f4f4d10ac7180fc1625999d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.123907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068de2d93f4f4d10ac7180fc1625999d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.124589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068de2d93f4f4d10ac7180fc1625999d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.132351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068de2d93f4f4d10ac7180fc1625999d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.148626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1053/2068] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:45 2022] DELETE /identity/v3/users/068de2d93f4f4d10ac7180fc1625999d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:54:46.154449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.171934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.171934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.174998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:46.176497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.250512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1016/2069] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:46.261024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:46.263511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.329137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce310b-e7d7-4709-803a-8089d602f936 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.333333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.333975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.334252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.334526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.344613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.346868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5936adc-0d14-4dd9-a379-a38caa701682 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:46.361899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1054/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:46.550618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:46.553125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.567753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40eb06-1dd0-4383-a61e-f21c9071e063 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.573414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.574388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.574726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.574997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] PATH_INFO: `/v3/projects/bb0e8c31f64c4da1ae4ad2ca1f6c98be` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.576659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0e8c31f64c4da1ae4ad2ca1f6c98be)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.578633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:46.626715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0e8c31f64c4da1ae4ad2ca1f6c98be', 'request_id': 'req-f23fddcb-fdce-4f38-a312-775c634eafcd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.632852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0e8c31f64c4da1ae4ad2ca1f6c98be', 'request_id': 'req-f23fddcb-fdce-4f38-a312-775c634eafcd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.641580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0e8c31f64c4da1ae4ad2ca1f6c98be', 'request_id': 'req-f23fddcb-fdce-4f38-a312-775c634eafcd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.652104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] The token cache is being invalidate because project bb0e8c31f64c4da1ae4ad2ca1f6c98be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:54:46.652428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.653040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f23fddcb-fdce-4f38-a312-775c634eafcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:46.654743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1017/2071] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:54:46 2022] DELETE /identity/v3/projects/bb0e8c31f64c4da1ae4ad2ca1f6c98be => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:46.659877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.660192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.660512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.678416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:46.679629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.739839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1055/2072] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:46.747672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.748087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.748719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.775047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:46.776439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:46.788564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1018/2073] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:46.793263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:46.803209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731d5c0-6873-4e08-81dd-89888b49679f None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.808423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.809082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.809347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.809620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.810667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:54:46.811208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.812857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:46.824994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1056/2074] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:54:46.829539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:46.839208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990add-2a80-47de-a31e-7c02a1a05942 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.845068 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.845857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.846140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.846412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.847120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.849071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:46.876307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1019/2075] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:54:46 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:46.879525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:46.890035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc69d-a535-4b4b-9bb2-4f0c012e7e6a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:46.970072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:46.970916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:46.971206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:46.971482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:46.972849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:46.975353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:47.004419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1057/2076] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:54:46 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:54:47.007600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:47.018981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a2d99-8527-4f35-9403-475bb674cab2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:47.025397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:47.032326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:47.032326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:47.032326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:47.032326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:47.032326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:47.046370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1020/2077] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:47.047853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:47.059043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d720b-b20e-4b74-8161-af1dc58405f5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:47.064229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:47.064933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:47.065215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:47.065486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:47.066412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:47.068007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717956fb-bf99-4283-b2aa-844d6c3b2a9c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:47.085604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1058/2078] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:54:47.092696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:47.111832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a2cb6-499f-4978-9cc3-750acdada509 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:47.119378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:47.120910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:47.121347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:47.121800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] PATH_INFO: `/v3/projects/5eab5b22675c4c7bacddb9659bed29c2/users/2b6a0a26494e4e619d504f10ef20fb3b/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:47.141301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eab5b22675c4c7bacddb9659bed29c2, user_id=2b6a0a26494e4e619d504f10ef20fb3b, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:47.149321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:47.165742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1021/2079] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:54:47 2022] PUT /identity/v3/projects/5eab5b22675c4c7bacddb9659bed29c2/users/2b6a0a26494e4e619d504f10ef20fb3b/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:47.171531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6604-eabd-4994-a3da-5e0082742216 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:47.172018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6604-eabd-4994-a3da-5e0082742216 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:47.172584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6604-eabd-4994-a3da-5e0082742216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:47.193068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-eb3b6604-eabd-4994-a3da-5e0082742216 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:47.194226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-eb3b6604-eabd-4994-a3da-5e0082742216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:47.268950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1059/2080] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:47.275979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:47.277780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:47.342061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de59906-416a-4fe7-8583-f7306f9b1be4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:47.346126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:47.347054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:47.347794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:47.348231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:47.360903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:47.363199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:47.375098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1022/2081] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:48.035804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ea6-c364-4524-a173-66663da46023 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:48.036446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ea6-c364-4524-a173-66663da46023 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:48.036446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ea6-c364-4524-a173-66663da46023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:48.074252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-87709ea6-c364-4524-a173-66663da46023 None None] MFA Rules not processed for user `2b6a0a26494e4e619d504f10ef20fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:48.075541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-87709ea6-c364-4524-a173-66663da46023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:48.152412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1060/2082] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:48.164983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:48.166816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:48.240732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77971e-bd41-4f91-9f33-cb1337204b2c service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:48.245690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:48.246691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:48.247107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:48.247542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:48.259535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:48.261694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:48.274292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1023/2083] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:54:53.827108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848a37-81fd-4ca5-b910-757cdb40d194 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:53.827673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848a37-81fd-4ca5-b910-757cdb40d194 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:53.827673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6848a37-81fd-4ca5-b910-757cdb40d194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:53.848804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c6848a37-81fd-4ca5-b910-757cdb40d194 None None] MFA Rules not processed for user `2b6a0a26494e4e619d504f10ef20fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:53.850503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c6848a37-81fd-4ca5-b910-757cdb40d194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:53.863258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1061/2084] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:59.418180 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:59.419864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:59.434944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c82fe-8dd7-4bda-acfb-14718114e546 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:59.440721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:59.441605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:59.441880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:59.442155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] PATH_INFO: `/v3/users/aa4e72d435db458b8490965138dad88e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:59.443589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4e72d435db458b8490965138dad88e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:59.445571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:59.500652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4e72d435db458b8490965138dad88e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:59.513549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4e72d435db458b8490965138dad88e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:59.520774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4e72d435db458b8490965138dad88e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:59.521481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4e72d435db458b8490965138dad88e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:59.531019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4e72d435db458b8490965138dad88e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:54:59.537415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1024/2085] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:54:59 2022] DELETE /identity/v3/users/aa4e72d435db458b8490965138dad88e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:54:59.542034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:59.542513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:59.542611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:59.559851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:54:59.561064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:59.620694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1062/2086] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:54:59.630701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:54:59.632617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:54:59.697245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8688f0-453d-4431-b887-88b035788c9f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:54:59.703756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:54:59.704445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:54:59.704727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:54:59.705003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:54:59.715159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:54:59.716781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294d389-4dd7-4e4f-9a70-cf2eccd3b295 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:54:59.726665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1025/2087] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:55:00.007807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.009283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:00.021842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37259c5-8344-4c97-a7b4-c39a54efa90c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.031551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.032782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.033278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.033984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] PATH_INFO: `/v3/projects/a7981466c1984f26a302a2498dfb7d36` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.035431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7981466c1984f26a302a2498dfb7d36)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.037482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.105392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7981466c1984f26a302a2498dfb7d36', 'request_id': 'req-846b04aa-4a5a-44a3-a985-c2117061a63a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:00.112798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7981466c1984f26a302a2498dfb7d36', 'request_id': 'req-846b04aa-4a5a-44a3-a985-c2117061a63a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:00.122169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7981466c1984f26a302a2498dfb7d36', 'request_id': 'req-846b04aa-4a5a-44a3-a985-c2117061a63a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:00.132640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] The token cache is being invalidate because project a7981466c1984f26a302a2498dfb7d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:55:00.132934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:00.133576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-846b04aa-4a5a-44a3-a985-c2117061a63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:00.135173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1063/2088] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:55:00 2022] DELETE /identity/v3/projects/a7981466c1984f26a302a2498dfb7d36 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:00.140270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.140568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.140994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.158996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:00.160183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:00.237109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1026/2089] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:00.245994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.246281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.246557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.266356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:00.267597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:00.275541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1064/2090] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:00.280228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.290372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d79bd7-de1d-478f-8669-84f8b9a8a374 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.295039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.295690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.295965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.296234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.297157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:55:00.297466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.299224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.315621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1027/2091] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:55:00.319750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.332697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4c8c-430f-4985-b185-f8074b62111a None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.337467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.338128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.338404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.339437 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.340396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.342291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.374005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1065/2092] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:55:00 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:00.375323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.385365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef1ea-263e-4d5d-a0de-87baab3f776e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.441331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.442050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.442339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.442616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.443303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.445317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.469266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1028/2093] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:55:00 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:00.473504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.483853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67906b0e-df0e-4918-95d3-c1fca77ff5b6 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.489010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.489745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.490095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.490439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.491238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.492888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.504338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1066/2094] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:55:00.508615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.518919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7120c-669e-47fe-bf7f-5f783ec3a2dc admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.523527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.524241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.524548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.524821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.525579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.527175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85341c4-55f4-4cfc-9b92-eaa3fb22f6fd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.537245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1029/2095] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:55:00.541421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.552840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885fca0-3cae-4cb2-b59e-0bdcf1788164 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.557499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.558378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.558653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.558931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] PATH_INFO: `/v3/projects/a00cd6b85252457abfea8cdc68dbbebe/users/9e29b068b9b34827be27bb4f83bae523/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.574611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00cd6b85252457abfea8cdc68dbbebe, user_id=9e29b068b9b34827be27bb4f83bae523, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.577343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.591314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1067/2096] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:00 2022] PUT /identity/v3/projects/a00cd6b85252457abfea8cdc68dbbebe/users/9e29b068b9b34827be27bb4f83bae523/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:00.596698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa515e-e478-45ea-9b77-e1bbe7cd3050 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.596982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa515e-e478-45ea-9b77-e1bbe7cd3050 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.597310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa515e-e478-45ea-9b77-e1bbe7cd3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.617152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b3aa515e-e478-45ea-9b77-e1bbe7cd3050 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:00.618346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b3aa515e-e478-45ea-9b77-e1bbe7cd3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:00.678551 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1030/2097] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:55:00.688470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:00.690319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:00.749228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99336c0d-ec4e-4790-a602-5d466dc1fa5f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:00.753358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:00.753999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:00.754280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:00.754554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:00.763886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:00.765374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:00.775184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1068/2098] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:55:06.096320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.097289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.097821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.126619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] MFA Rules not processed for user `9e29b068b9b34827be27bb4f83bae523`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:06.127869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:06.196206 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1031/2099] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:55:06.201036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.211511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0490f3-a0bb-4f9e-8791-1bcd408fe467 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.217023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.217673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.217957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.218237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.218973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.220989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.251153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1069/2100] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:55:06 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:06.255289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.266442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bed775-fcd1-462f-ac99-c5dc2917a357 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.330731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.331513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.331865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.332220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.333047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.335088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.368084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1032/2101] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:55:06 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:06.370197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.383851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceb032-5697-4e5d-a5c7-1f643b3b3fbc admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.392213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.393216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.393503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.393781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.394526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.396190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.413899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1070/2102] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:06.415864 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.426865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945af7-bbc9-47ec-a1e2-bf8b230af1c0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.432544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.433556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.433907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.434286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] PATH_INFO: `/v3/projects/b320f1895ceb4ee0b3478453c108f788/users/d6aaee3409e04fe18e90733809be20df/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.450387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b320f1895ceb4ee0b3478453c108f788, user_id=d6aaee3409e04fe18e90733809be20df, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.453353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.467503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1033/2103] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:06 2022] PUT /identity/v3/projects/b320f1895ceb4ee0b3478453c108f788/users/d6aaee3409e04fe18e90733809be20df/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:06.472302 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.482089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fc8b-340b-466b-bb6b-b9857a381cc9 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.540135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.540775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.541063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.541345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.542081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.543720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d5676-3f37-4a34-a4b9-cee9dc4e4ddc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.554465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1071/2104] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:06.558926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:06.570046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afafbc-1ecc-420a-a175-dd303982fce9 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:06.575211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:06.576092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:06.576411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:06.576687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] PATH_INFO: `/v3/projects/b320f1895ceb4ee0b3478453c108f788/users/d6aaee3409e04fe18e90733809be20df/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:06.579083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b320f1895ceb4ee0b3478453c108f788, user_id=d6aaee3409e04fe18e90733809be20df, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:06.581777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:06.593344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1034/2105] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:06 2022] PUT /identity/v3/projects/b320f1895ceb4ee0b3478453c108f788/users/d6aaee3409e04fe18e90733809be20df/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:12.037193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332258b-c599-4ee5-878a-1efcb4e579e3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:12.038153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332258b-c599-4ee5-878a-1efcb4e579e3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:12.038652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332258b-c599-4ee5-878a-1efcb4e579e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:12.087362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5332258b-c599-4ee5-878a-1efcb4e579e3 None None] MFA Rules not processed for user `d6aaee3409e04fe18e90733809be20df`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:12.088644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5332258b-c599-4ee5-878a-1efcb4e579e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:12.197561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1072/2106] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:12.210137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:12.212446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:12.317517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd91e-29b5-46f3-88b1-1983b082ffa6 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:12.321561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:12.322239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:12.322516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:12.322792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:12.387232 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:12.388758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:12.398783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1035/2107] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:55:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:55:12.500922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094e58f-2a47-474d-897b-7b2a1a323b7f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:12.501324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094e58f-2a47-474d-897b-7b2a1a323b7f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:12.501595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094e58f-2a47-474d-897b-7b2a1a323b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:12.529212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d094e58f-2a47-474d-897b-7b2a1a323b7f None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:12.530500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d094e58f-2a47-474d-897b-7b2a1a323b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:12.594592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1073/2108] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:55:12.618965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:12.624030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:12.707722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde90f4-a963-4b62-bd5c-289c6df2d974 service glance] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:12.713524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:12.714254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:12.714607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:12.714948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:12.724666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:12.726352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84776dee-2e3c-4d1e-825c-1e527eb0c958 service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:12.735857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1036/2109] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:55:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:55:22.390530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:22.391701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:22.391701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:22.410839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] MFA Rules not processed for user `9e29b068b9b34827be27bb4f83bae523`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:22.412046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:22.420338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1074/2110] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:30.117684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:30.118685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:30.119284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:30.160203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:30.161574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:30.170124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1037/2111] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:55:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:30.186060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:30.199000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c526b8c-91e3-4d20-9678-38f44dc8dbd8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:30.204499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:30.205158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:30.205557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:30.205837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:30.215891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:30.217542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b6e4d-5565-4bf3-91b9-52dd41df05cc service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:30.226296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1075/2112] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:55:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:55:31.061016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:31.062571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:31.172279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9698adc-ec75-45dd-beb6-7782bc4b3818 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:31.177438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:31.178686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:31.178972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:31.179244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] PATH_INFO: `/v3/users/8b24fdd8295f46a0af75d45a5e3f51a7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:31.180587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b24fdd8295f46a0af75d45a5e3f51a7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:31.182553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:31.233120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b24fdd8295f46a0af75d45a5e3f51a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.249326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b24fdd8295f46a0af75d45a5e3f51a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.256790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b24fdd8295f46a0af75d45a5e3f51a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.257526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b24fdd8295f46a0af75d45a5e3f51a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.267699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b24fdd8295f46a0af75d45a5e3f51a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.275742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1038/2113] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:55:31 2022] DELETE /identity/v3/users/8b24fdd8295f46a0af75d45a5e3f51a7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:31.280292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:31.280968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:31.280968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:31.315811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:31.320118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:31.390253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1076/2114] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:31.401722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:31.403396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:31.484072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e58828-9990-4b71-bc4e-87ce59ebadda admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:31.488561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:31.489315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:31.489596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:31.489867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:31.499358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:31.500839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:31.511330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1039/2115] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:55:31.814168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:31.815959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:31.829820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26207c9-036b-4df1-8445-db0295d8b866 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:31.834909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:31.836507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:31.836803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:31.837076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] PATH_INFO: `/v3/projects/e4111340ce99406d8ed0502637a0baee` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:31.838830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4111340ce99406d8ed0502637a0baee)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:31.840959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:31.890129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4111340ce99406d8ed0502637a0baee', 'request_id': 'req-91f2608a-6610-429e-b161-72f012aa3c72'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.896360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4111340ce99406d8ed0502637a0baee', 'request_id': 'req-91f2608a-6610-429e-b161-72f012aa3c72'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.904544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4111340ce99406d8ed0502637a0baee', 'request_id': 'req-91f2608a-6610-429e-b161-72f012aa3c72'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.915889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] The token cache is being invalidate because project e4111340ce99406d8ed0502637a0baee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:55:31.916191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.916860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:31.918700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1077/2116] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:55:31 2022] DELETE /identity/v3/projects/e4111340ce99406d8ed0502637a0baee => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:31.922702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:31.932738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7acb1f-44af-439a-9dba-6a34c08f8130 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:31.990902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:31.991824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:31.992102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:31.992420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] PATH_INFO: `/v3/users/995bd1bd26fe4b3c87d3018484655703` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:31.993646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=995bd1bd26fe4b3c87d3018484655703)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:31.995593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:32.044885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995bd1bd26fe4b3c87d3018484655703'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.056663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995bd1bd26fe4b3c87d3018484655703'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.063316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995bd1bd26fe4b3c87d3018484655703'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.064042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995bd1bd26fe4b3c87d3018484655703'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.075875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-5b31879b-5d8d-430d-baac-cd6a45ef1ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995bd1bd26fe4b3c87d3018484655703'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.083858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1040/2117] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:55:31 2022] DELETE /identity/v3/users/995bd1bd26fe4b3c87d3018484655703 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:32.439766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:32.441540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:32.455312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2608a-6610-429e-b161-72f012aa3c72 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:32.531312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:32.532217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.532633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.532914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] PATH_INFO: `/v3/projects/26c7796dc2054065a9275b1a58feee6c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.534108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c7796dc2054065a9275b1a58feee6c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:32.536531 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:32.589703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c7796dc2054065a9275b1a58feee6c', 'request_id': 'req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.596589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c7796dc2054065a9275b1a58feee6c', 'request_id': 'req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.607859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c7796dc2054065a9275b1a58feee6c', 'request_id': 'req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.619511 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] The token cache is being invalidate because project 26c7796dc2054065a9275b1a58feee6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:55:32.619796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.620489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-033d9dd8-9d5d-4928-b078-bc61b3ba5c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:55:32.624513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1078/2118] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:55:32 2022] DELETE /identity/v3/projects/26c7796dc2054065a9275b1a58feee6c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:32.629637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.630204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.630327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.678903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:32.680129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:32.747049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1041/2119] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:55:32.758239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e082b2-530c-4514-a341-07568545bb2b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.758239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e082b2-530c-4514-a341-07568545bb2b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.758239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e082b2-530c-4514-a341-07568545bb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.774281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-24e082b2-530c-4514-a341-07568545bb2b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:32.775605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-24e082b2-530c-4514-a341-07568545bb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:32.783502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1079/2120] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:32.788026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:32.798125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2a280-14b6-4839-94b0-38e8f233303a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:32.803211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:32.803854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.804296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.804624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.805387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:55:32.805732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:32.807359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:32.819216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1042/2121] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:55:32.823935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e082b2-530c-4514-a341-07568545bb2b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:32.835171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e082b2-530c-4514-a341-07568545bb2b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:32.840157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:32.840850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.841135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.841407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.842110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:32.844052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:32.872516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1080/2122] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:55:32 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:32.877099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:32.888364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93157-d089-46fe-a1e2-94358f231980 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:32.949764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:32.950443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.950728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.951004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.951697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:32.953666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:32.976558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1043/2123] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:55:32 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:32.980756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:32.992318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216a3f-be94-4c8c-9c54-130b32708dec admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:32.997538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:32.998252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:32.998536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:32.998833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:32.999565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.001199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.013066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1081/2124] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:33.017204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.028904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798f47-1b38-449a-aa21-8d3c0633375f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.034083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.034757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.035045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.035318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.036795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.038420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0958-6316-4862-9878-0380b8fc801d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.049906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1044/2125] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:33.054774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.065545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76047321-65c4-47b4-8374-aab8ccd48500 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.071136 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.072073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.072380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.072653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] PATH_INFO: `/v3/projects/7c0803f960ad457ba7590d8a324998a6/users/89970cc6751247a88c4401d3cc3beddf/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.088789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0803f960ad457ba7590d8a324998a6, user_id=89970cc6751247a88c4401d3cc3beddf, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.091456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.106716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1082/2126] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:33 2022] PUT /identity/v3/projects/7c0803f960ad457ba7590d8a324998a6/users/89970cc6751247a88c4401d3cc3beddf/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:33.118450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.118741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.119023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.147310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] MFA Rules not processed for user `89970cc6751247a88c4401d3cc3beddf`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:33.148640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:33.216017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1045/2127] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:55:33.220680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.294521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a0fa-b4a6-4d03-8eea-dd280711fba2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.300880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.302147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.302878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.303360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.304295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.306770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.334239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1083/2128] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:55:33 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:55:33.338222 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.353522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110101b-e6fe-493e-8106-be414dc8d14f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.417065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.417943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.418380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.418676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.419519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.423695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.447025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1046/2129] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:55:33 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:55:33.451089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.464280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de704f87-232a-4cb0-99d4-2d52631fe590 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.469306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.470043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.470315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.470592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.471335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.473019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.484612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1084/2130] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:33.489644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.500111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b02a-4a6f-47f7-8c33-73b1524360a3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.505712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.506613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.506896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.507170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] PATH_INFO: `/v3/projects/d981a59c699b475bb39123db1a3621db/users/450365e7bcbf44ed9034c952384680fd/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.522235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d981a59c699b475bb39123db1a3621db, user_id=450365e7bcbf44ed9034c952384680fd, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.524957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.539603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1047/2131] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:33 2022] PUT /identity/v3/projects/d981a59c699b475bb39123db1a3621db/users/450365e7bcbf44ed9034c952384680fd/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:33.543712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.553764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518d53b-cbbe-45f5-bc48-6c49e9d4d104 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.620453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.621525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.621649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.621900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.622709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.624544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c379984-2a33-4cb8-9ddb-f232d7a660dc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.636365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1085/2132] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:55:33.640356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.651544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ff07-4d7b-4afe-8ea9-3a8e17b23227 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.664981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.667091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.667460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.667854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] PATH_INFO: `/v3/projects/d981a59c699b475bb39123db1a3621db/users/450365e7bcbf44ed9034c952384680fd/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.670494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d981a59c699b475bb39123db1a3621db, user_id=450365e7bcbf44ed9034c952384680fd, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.673502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:33.686396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1048/2133] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:55:33 2022] PUT /identity/v3/projects/d981a59c699b475bb39123db1a3621db/users/450365e7bcbf44ed9034c952384680fd/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:55:33.696930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.697277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.697565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.728365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] MFA Rules not processed for user `450365e7bcbf44ed9034c952384680fd`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:55:33.729893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:33.808815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1086/2134] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:55:33.820181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:33.822072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:55:33.890477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc022-4a6e-43bd-b974-f7fc1950c780 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:33.900981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:33.902599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:33.906764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:33.907817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:33.994794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:33.996165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:34.005633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1049/2135] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:55:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:55:45.894270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:55:45.914411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355d3f-40b8-4096-bc60-a7b7aa6f159d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:55:45.918368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:55:45.919017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:55:45.919300 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:55:45.919572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:55:45.928906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:55:45.930381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc84a7c-9b3d-493b-a5c6-a82659651f0b service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:55:45.940935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1087/2136] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:55:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:15.122375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:15.124158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:15.218664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265e79a-888b-4aaa-91ca-b17daf96786b service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:15.224919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:15.225740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:15.226022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:15.226840 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:15.228819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:15.246287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:15.247938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:15.259360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1050/2137] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:15.675298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf769fc-63cc-4d01-8288-3ce8d9ce6b6e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:15.675845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf769fc-63cc-4d01-8288-3ce8d9ce6b6e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:15.675962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf769fc-63cc-4d01-8288-3ce8d9ce6b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:15.699208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1cf769fc-63cc-4d01-8288-3ce8d9ce6b6e None None] MFA Rules not processed for user `5db47a076f5d4650b0a7a6945ae70a52`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:15.700470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1cf769fc-63cc-4d01-8288-3ce8d9ce6b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:15.757153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1088/2138] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 00:56:18.150699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:18.217121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b1a6d-707c-4bf6-bbcf-115bbabfba00 service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:18.222175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:18.223113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:18.223369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:18.223665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] PATH_INFO: `/v3/users/89970cc6751247a88c4401d3cc3beddf` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:18.224898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89970cc6751247a88c4401d3cc3beddf)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:18.226845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:18.321242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89970cc6751247a88c4401d3cc3beddf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:18.335054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89970cc6751247a88c4401d3cc3beddf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:18.350303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89970cc6751247a88c4401d3cc3beddf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:18.351181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89970cc6751247a88c4401d3cc3beddf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:18.363525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89970cc6751247a88c4401d3cc3beddf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:18.370719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1051/2139] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:18 2022] DELETE /identity/v3/users/89970cc6751247a88c4401d3cc3beddf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:18.375796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:18.376792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:18.377243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:18.397326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:18.400827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:18.501973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1089/2140] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:18.509195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:18.511050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:18.574148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a08ff-3d9a-4b2d-b39f-edd821233f48 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:18.579904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:18.580649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:18.581017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:18.581360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:18.590807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:18.592476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:18.602763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1052/2141] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:18.941844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:18.944966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:18.964326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a118bc9-27a5-4a6d-bd33-fd70c77f1ea5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:18.972491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:18.973765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:18.986544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:18.986859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] PATH_INFO: `/v3/projects/7c0803f960ad457ba7590d8a324998a6` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:18.988353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0803f960ad457ba7590d8a324998a6)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:18.992761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:19.047890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0803f960ad457ba7590d8a324998a6', 'request_id': 'req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.054727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0803f960ad457ba7590d8a324998a6', 'request_id': 'req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.066268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0803f960ad457ba7590d8a324998a6', 'request_id': 'req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.082143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] The token cache is being invalidate because project 7c0803f960ad457ba7590d8a324998a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:19.082432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.083316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.085042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1090/2142] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:18 2022] DELETE /identity/v3/projects/7c0803f960ad457ba7590d8a324998a6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:19.089146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:19.099898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238efc-53f5-4373-aa8d-87a635c76bc5 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:19.193773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:19.194678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.194959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.195230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] PATH_INFO: `/v3/users/450365e7bcbf44ed9034c952384680fd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.196496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=450365e7bcbf44ed9034c952384680fd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:19.198462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:19.257838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450365e7bcbf44ed9034c952384680fd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.272072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450365e7bcbf44ed9034c952384680fd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.279122 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450365e7bcbf44ed9034c952384680fd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.280787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450365e7bcbf44ed9034c952384680fd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.292400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba4a6153-9a77-49be-8a23-491a51cbd7cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450365e7bcbf44ed9034c952384680fd'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.299956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1053/2143] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:19 2022] DELETE /identity/v3/users/450365e7bcbf44ed9034c952384680fd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:19.581929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:19.583447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:19.599067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77359ef4-adcf-4fbe-99b2-cfe35a7819c1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:19.657892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:19.658705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.658972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.659244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] PATH_INFO: `/v3/projects/d981a59c699b475bb39123db1a3621db` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.660467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d981a59c699b475bb39123db1a3621db)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:19.662405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:19.710862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd981a59c699b475bb39123db1a3621db', 'request_id': 'req-c4831a38-e78d-4e20-bdad-2e15d82edbba'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.716944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd981a59c699b475bb39123db1a3621db', 'request_id': 'req-c4831a38-e78d-4e20-bdad-2e15d82edbba'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.728052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd981a59c699b475bb39123db1a3621db', 'request_id': 'req-c4831a38-e78d-4e20-bdad-2e15d82edbba'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.742800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] The token cache is being invalidate because project d981a59c699b475bb39123db1a3621db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:19.743071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.743759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c4831a38-e78d-4e20-bdad-2e15d82edbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:19.745385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1091/2144] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:19 2022] DELETE /identity/v3/projects/d981a59c699b475bb39123db1a3621db => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:19.751474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.751762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.752041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.769520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:19.770770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:19.830592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1054/2145] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:19.842876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.843268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.843518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.863559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:19.864843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:19.873792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1092/2146] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:19.877946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:19.887728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25bf8c-c24b-4ff7-9350-9a1ab40166ec None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:19.893279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:19.893930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.894216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.894501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.895241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:56:19.895546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:19.897201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:19.909384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1055/2147] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:56:19.913875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:19.923554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf34f-b63b-4813-8142-446d8fc2ebfb None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:19.928965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:19.929683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:19.929970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:19.930241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:19.930951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:19.932932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:19.961327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1093/2148] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:56:19 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:19.965883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:19.976112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6374ab-96a3-455f-a507-7bfa9342402f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.033417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.033966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.034241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.034531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.035226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.037312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.058202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1056/2149] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:56:19 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:56:20.062302 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.073850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba86b4-0039-41ec-bd5f-eacf74029455 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.079469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.080127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.080442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.080716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.081449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.083035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.093666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1094/2150] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:56:20.097640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.107236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed320fb-48cb-4d89-87d0-937b9fb9a2cc admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.115100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.115759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.116035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.116339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.117041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.118664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24337e-90d7-4327-88c5-0028119d4fe5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.129124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1057/2151] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:20.133284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.143681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c7cd7-e867-48b0-8135-512fcaf1390d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.148383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.149267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.149573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.150122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] PATH_INFO: `/v3/projects/6dd4b90172ee40ef955e362d5662155e/users/699ba5ee83bf4aa18ac948ccb208a635/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.164676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd4b90172ee40ef955e362d5662155e, user_id=699ba5ee83bf4aa18ac948ccb208a635, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.167309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.181634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1095/2152] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:20 2022] PUT /identity/v3/projects/6dd4b90172ee40ef955e362d5662155e/users/699ba5ee83bf4aa18ac948ccb208a635/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:20.192204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.192521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.192800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.220842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] MFA Rules not processed for user `699ba5ee83bf4aa18ac948ccb208a635`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:20.222038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:20.298573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1058/2153] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:20.303102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.366070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d359c0-c202-438f-8c0b-353f77c8b230 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.371641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.372288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.372596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.372871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.373598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.375480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.412569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1096/2154] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:20.418584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.433819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1b19a-99e0-4d96-9dd6-d200492b2a7b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.495580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.496368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.496647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.496916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.497634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.499597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.521173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1059/2155] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:20.525381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.536472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401482-85b4-472a-bd0d-477d4ec53641 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.541805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.542466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.542750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.542994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.543734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.545364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.556473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1097/2156] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:20.560677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.570904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b534d2c-6880-4147-873c-d12950ad7aba admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.576126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.577054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.577332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.577610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.578377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.579974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22e944-3e2c-466e-acce-53535c01bb75 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.590394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1060/2157] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:20.594606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.604522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31668-2d54-442d-a25d-9ff91140fcae admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.609614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.610487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.610798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.611043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] PATH_INFO: `/v3/projects/0d3dcc01a68344e6927d636fbe10d7ba/users/934cd70f4b9a44ffa0b7927aaf989f3c/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.625779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3dcc01a68344e6927d636fbe10d7ba, user_id=934cd70f4b9a44ffa0b7927aaf989f3c, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.628468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.646029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1098/2158] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:20 2022] PUT /identity/v3/projects/0d3dcc01a68344e6927d636fbe10d7ba/users/934cd70f4b9a44ffa0b7927aaf989f3c/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:20.657391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.657684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.657968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.684565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] MFA Rules not processed for user `934cd70f4b9a44ffa0b7927aaf989f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:20.685855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:20.758646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1061/2159] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:20.763789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.834317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32974-4aec-4c67-8360-49b2c2fbdef1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.839354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.840014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.840331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.840610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.841298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.843487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.874148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1099/2160] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:20.878241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.889098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01a978-791b-495b-9f61-bdcde0c817d4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.945934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.946605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.946887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.947160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.947837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.949797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:20.970463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1062/2161] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:56:20 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:20.974652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:20.985517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c894db5-ca4b-4da9-8999-3d8ede851b3a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:20.990709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:20.991365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:20.991649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:20.991920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:20.992676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:20.994233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.008519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1100/2162] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 00:56:21.009857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:21.019920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4ea6-854a-40ca-8d7a-5d52a382140a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:21.024793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:21.025676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.025955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.026212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] PATH_INFO: `/v3/projects/59b7dcc75c9241bcae6144f497b6ca70/users/51b4ddd2582641b4abb0199aced090b8/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.040834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b7dcc75c9241bcae6144f497b6ca70, user_id=51b4ddd2582641b4abb0199aced090b8, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:21.043503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.059942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1063/2163] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:21 2022] PUT /identity/v3/projects/59b7dcc75c9241bcae6144f497b6ca70/users/51b4ddd2582641b4abb0199aced090b8/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:21.064653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:21.077279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7db-a823-4f3c-87cb-cbba4e3089ba admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:21.136595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:21.137263 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.137548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.137819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.138541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:21.140521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af2cd6-838b-455d-80bd-42dee0d63788 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.150565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1101/2164] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:21.155014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:21.165415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac6775-4896-4186-92bc-a972ecabc060 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:21.170968 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:21.171881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.172163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.172476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] PATH_INFO: `/v3/projects/59b7dcc75c9241bcae6144f497b6ca70/users/51b4ddd2582641b4abb0199aced090b8/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.174627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b7dcc75c9241bcae6144f497b6ca70, user_id=51b4ddd2582641b4abb0199aced090b8, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:21.177311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.188312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1064/2165] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:21 2022] PUT /identity/v3/projects/59b7dcc75c9241bcae6144f497b6ca70/users/51b4ddd2582641b4abb0199aced090b8/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:21.198495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.198787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.199061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.227798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] MFA Rules not processed for user `51b4ddd2582641b4abb0199aced090b8`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:21.229042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:21.297751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1102/2166] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:21.309408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:21.311446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:21.386815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08fd07-fa7e-4adb-a5ef-6eca2837258b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:21.391286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:21.391934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.392216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.392529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.401873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:21.403337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.413098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1065/2167] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:21.458127 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:21.459619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:21.474069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7a02-aedf-4e1c-a5bd-ed20d10f0188 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:21.480420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:21.481392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:21.481810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:21.482173 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:21.539074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:21.540610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406e7b-8200-4bc8-b133-07dd38699e05 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:21.549907 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1103/2168] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:24.460573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:24.528214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678591f7-a25e-491a-bd42-88773e4d8e80 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:24.536309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:24.537634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:24.537985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:24.538319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] PATH_INFO: `/v3/users/699ba5ee83bf4aa18ac948ccb208a635` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:24.540402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=699ba5ee83bf4aa18ac948ccb208a635)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:24.543483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:24.598576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699ba5ee83bf4aa18ac948ccb208a635'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:24.611587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699ba5ee83bf4aa18ac948ccb208a635'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:24.618904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699ba5ee83bf4aa18ac948ccb208a635'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:24.619560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699ba5ee83bf4aa18ac948ccb208a635'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:24.628585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699ba5ee83bf4aa18ac948ccb208a635'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:24.634729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1066/2169] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:24 2022] DELETE /identity/v3/users/699ba5ee83bf4aa18ac948ccb208a635 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:24.639354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:24.639871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:24.639971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:24.656791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:24.658100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:24.717409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1104/2170] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:24.727603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:24.729112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:24.792909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deeb93-fb95-4a08-bc53-6f7e46951e7b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:24.797101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:24.797855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:24.798187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:24.798452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:24.807963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:24.809802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:24.823514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1067/2171] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:25.120690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:25.122245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:25.134585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750915d0-9bd0-49c0-8c56-0116959976c6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:25.140079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:25.141124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:25.141470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:25.141745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] PATH_INFO: `/v3/projects/6dd4b90172ee40ef955e362d5662155e` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:25.142985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd4b90172ee40ef955e362d5662155e)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:25.144994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:25.195771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd4b90172ee40ef955e362d5662155e', 'request_id': 'req-07b20daa-ca5b-460a-a60a-e45754ade830'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.202074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd4b90172ee40ef955e362d5662155e', 'request_id': 'req-07b20daa-ca5b-460a-a60a-e45754ade830'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.212276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd4b90172ee40ef955e362d5662155e', 'request_id': 'req-07b20daa-ca5b-460a-a60a-e45754ade830'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.223844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] The token cache is being invalidate because project 6dd4b90172ee40ef955e362d5662155e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:25.224132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.224802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.226354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1105/2172] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:25 2022] DELETE /identity/v3/projects/6dd4b90172ee40ef955e362d5662155e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:25.230283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:25.240564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ee59f-ac8f-47a1-ba18-d77216ce93ca service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:25.295445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:25.296311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:25.296589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:25.296855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] PATH_INFO: `/v3/users/934cd70f4b9a44ffa0b7927aaf989f3c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:25.298127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] RBAC: Authorizing `identity:delete_user(user_id=934cd70f4b9a44ffa0b7927aaf989f3c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:25.300051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:25.349892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934cd70f4b9a44ffa0b7927aaf989f3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.362698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934cd70f4b9a44ffa0b7927aaf989f3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.369509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934cd70f4b9a44ffa0b7927aaf989f3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.370284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934cd70f4b9a44ffa0b7927aaf989f3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.379602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934cd70f4b9a44ffa0b7927aaf989f3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.387268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1068/2173] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:25 2022] DELETE /identity/v3/users/934cd70f4b9a44ffa0b7927aaf989f3c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:25.698279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:25.699856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:25.713307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20daa-ca5b-460a-a60a-e45754ade830 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:25.773088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:25.773966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:25.774248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:25.774507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] PATH_INFO: `/v3/projects/0d3dcc01a68344e6927d636fbe10d7ba` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:25.775775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3dcc01a68344e6927d636fbe10d7ba)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:25.777741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:25.831088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3dcc01a68344e6927d636fbe10d7ba', 'request_id': 'req-4f09475f-f439-409e-a82a-c1dfe1ff8edc'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.837225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3dcc01a68344e6927d636fbe10d7ba', 'request_id': 'req-4f09475f-f439-409e-a82a-c1dfe1ff8edc'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.845981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3dcc01a68344e6927d636fbe10d7ba', 'request_id': 'req-4f09475f-f439-409e-a82a-c1dfe1ff8edc'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.856930 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] The token cache is being invalidate because project 0d3dcc01a68344e6927d636fbe10d7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:25.857242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.857920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:25.859971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1106/2174] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:25 2022] DELETE /identity/v3/projects/0d3dcc01a68344e6927d636fbe10d7ba => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:25.864106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:25.865614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:25.878855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e180a0-f048-430f-88b6-c6ef4ab2480a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:25.937522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:25.939208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:25.939495 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:25.939774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] PATH_INFO: `/v3/users/51b4ddd2582641b4abb0199aced090b8` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:25.941349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b4ddd2582641b4abb0199aced090b8)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:25.943237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:25.996656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b4ddd2582641b4abb0199aced090b8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.009748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b4ddd2582641b4abb0199aced090b8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.016548 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b4ddd2582641b4abb0199aced090b8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.017407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b4ddd2582641b4abb0199aced090b8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.025920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9be0f5fb-7d5c-46e9-859a-1960ec612f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b4ddd2582641b4abb0199aced090b8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.032149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1069/2175] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:25 2022] DELETE /identity/v3/users/51b4ddd2582641b4abb0199aced090b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 00:56:26.311483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.313250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:26.327391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09475f-f439-409e-a82a-c1dfe1ff8edc admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.384175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.385105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.385378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.385644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] PATH_INFO: `/v3/projects/59b7dcc75c9241bcae6144f497b6ca70` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.386892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b7dcc75c9241bcae6144f497b6ca70)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.388858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.439241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b7dcc75c9241bcae6144f497b6ca70', 'request_id': 'req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.445341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b7dcc75c9241bcae6144f497b6ca70', 'request_id': 'req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.454927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b7dcc75c9241bcae6144f497b6ca70', 'request_id': 'req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.472931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] The token cache is being invalidate because project 59b7dcc75c9241bcae6144f497b6ca70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:26.473234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.473993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ef2f7d89-c26d-46e2-a9fd-fb4c321e4dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:26.475724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1107/2176] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:26 2022] DELETE /identity/v3/projects/59b7dcc75c9241bcae6144f497b6ca70 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:26.482001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.482289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.482581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.501545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:26.502772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:26.565362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1070/2177] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:26.574732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.575162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.575644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.596017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:26.597352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:26.606404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1108/2178] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:26.611241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.621496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60da56-46de-41ed-9eeb-12ecdb66b23f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.626784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.627431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.627711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.627983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.628742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:56:26.629040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.630635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.643265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1071/2179] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 00:56:26.648230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.662482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95432287-8e8d-4faf-8d97-8cbae6cd2525 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.670679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.671344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.671600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.671869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.672783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.674883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.704878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1109/2180] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:56:26 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:26.710555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.721531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13999b0e-5471-4d59-a05b-b319f20685d2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.779855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.780595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.780879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.781146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.781941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.784538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.805910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1072/2181] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:56:26 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:26.810414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.821058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aea8e-5d80-4a33-9dc1-1b44481aa68b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.826071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.826719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.826999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.827270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.827997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.829925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.840650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1110/2182] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:26.845283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.856283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660e4f8-8ecd-485f-be3c-78127e57af9d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.862926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.863561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.863868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.864131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.864897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.866497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949f459-2aed-4c40-9d65-8b6c2e6f35ba admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.878688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1073/2183] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:26.883886 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:26.900761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cca83-d76b-4324-86e4-4fcf5214ae98 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:26.907873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:26.908774 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.909050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.909320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] PATH_INFO: `/v3/projects/9fa48f9d954a4e5296eee8e8af1dbfb0/users/9328d578c1f24f169c2f1dc5fd0599ad/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.925388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa48f9d954a4e5296eee8e8af1dbfb0, user_id=9328d578c1f24f169c2f1dc5fd0599ad, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:26.928031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:26.944109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1111/2184] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:26 2022] PUT /identity/v3/projects/9fa48f9d954a4e5296eee8e8af1dbfb0/users/9328d578c1f24f169c2f1dc5fd0599ad/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:26.950207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0345f2-1343-4692-b446-8f3f81f26448 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:26.950618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0345f2-1343-4692-b446-8f3f81f26448 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:26.951034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0345f2-1343-4692-b446-8f3f81f26448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:26.976842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ba0345f2-1343-4692-b446-8f3f81f26448 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:26.977974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ba0345f2-1343-4692-b446-8f3f81f26448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:27.041356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1074/2185] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:27.051851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:27.053582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:27.113386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0abe-771f-4d07-a4f1-bef4106cc9e1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:27.117089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:27.117739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:27.118014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:27.118284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:27.127600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:27.129090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:27.138472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1112/2186] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:27.734625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:27.735680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:27.735680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:27.762417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] MFA Rules not processed for user `9328d578c1f24f169c2f1dc5fd0599ad`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:27.763736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:27.834384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1075/2187] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:27.840963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:27.853703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0b2d0-0c57-4434-95a2-103498f9ade2 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:27.861986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:27.863512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:27.864061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:27.864601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:27.865801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:27.869158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:27.908895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1113/2188] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:56:27 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:27.910280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:27.925694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12512867-8ed4-43d6-bdc4-776028bc993f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:27.984668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:27.985423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:27.985824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:27.985965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:27.986678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:27.988739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:28.009261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1076/2189] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:56:27 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:56:28.013234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:28.023620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c3f91-af2a-4f66-8ea8-24255e00b7df admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:28.028678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:28.029368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:28.029652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:28.029931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:28.030650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:28.032569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:28.043023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1114/2190] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:28.047540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:28.058268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03082ded-dd69-44da-b2c6-31e4539d84e2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:28.064385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:28.065297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:28.065573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:28.065845 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] PATH_INFO: `/v3/projects/77f3dc9200394ff1a5f17b523ff9537f/users/7596fb15151e481e9e324d13f3a21cd3/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:28.085974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f3dc9200394ff1a5f17b523ff9537f, user_id=7596fb15151e481e9e324d13f3a21cd3, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:28.090193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:28.110228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1077/2191] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:28 2022] PUT /identity/v3/projects/77f3dc9200394ff1a5f17b523ff9537f/users/7596fb15151e481e9e324d13f3a21cd3/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:28.114447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:28.124372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2e616-15af-43e0-9f3f-a80544100bb3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:28.187006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:28.187901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:28.188498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:28.188920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:28.190073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:28.192711 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd17eb-7204-471d-b9cc-afaca7f48a28 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:28.203556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1115/2192] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:56:28.208053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:28.218580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee10b6-0b74-4c6d-99f0-8d8470a34ea0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:28.223817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:28.224765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:28.225072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:28.225344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] PATH_INFO: `/v3/projects/77f3dc9200394ff1a5f17b523ff9537f/users/7596fb15151e481e9e324d13f3a21cd3/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:28.227346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f3dc9200394ff1a5f17b523ff9537f, user_id=7596fb15151e481e9e324d13f3a21cd3, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:28.230058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:28.241955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1078/2193] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:56:28 2022] PUT /identity/v3/projects/77f3dc9200394ff1a5f17b523ff9537f/users/7596fb15151e481e9e324d13f3a21cd3/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:28.854925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:28.855445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:28.855445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:28.884036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] MFA Rules not processed for user `7596fb15151e481e9e324d13f3a21cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:56:28.885581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:28.956833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1116/2194] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:56:28.968400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:28.970306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:29.025358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3e4a-927c-476a-a3aa-82df9c056077 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:29.029572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:29.030143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:29.030424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:29.030693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:29.086996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:29.088539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:29.098639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1079/2195] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:31.510039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:31.520675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ccc77-2ec9-4b52-9a2d-1c72ef4e1678 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:31.525040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:31.525689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:31.525974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:31.526250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:31.536195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:31.537647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:31.547470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1117/2196] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:37.131831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:37.192757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca13e8-6653-46d3-9a8a-700fcd7bb49c service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:37.197530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:37.198189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:37.198467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:37.198736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:37.200952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:37.275604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:37.277387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:37.287896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1080/2197] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:56:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:56:59.297073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:59.370817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096153b-29c4-4e78-8f81-301621fa9492 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:59.377726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:59.378615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:59.378918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:59.379191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] PATH_INFO: `/v3/users/9328d578c1f24f169c2f1dc5fd0599ad` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:59.383752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9328d578c1f24f169c2f1dc5fd0599ad)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:59.387647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:59.444855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9328d578c1f24f169c2f1dc5fd0599ad'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.456510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9328d578c1f24f169c2f1dc5fd0599ad'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.457164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9328d578c1f24f169c2f1dc5fd0599ad'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.457885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9328d578c1f24f169c2f1dc5fd0599ad'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.464485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9328d578c1f24f169c2f1dc5fd0599ad'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.479690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1118/2198] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:59 2022] DELETE /identity/v3/users/9328d578c1f24f169c2f1dc5fd0599ad => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:59.677296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:59.689877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f16c66-562a-4a1c-94e6-487400eacece service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:59.755806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:59.756922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:59.757266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:59.757627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] PATH_INFO: `/v3/projects/9fa48f9d954a4e5296eee8e8af1dbfb0` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:59.759365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa48f9d954a4e5296eee8e8af1dbfb0)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:59.762090 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:59.809764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa48f9d954a4e5296eee8e8af1dbfb0', 'request_id': 'req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.816094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa48f9d954a4e5296eee8e8af1dbfb0', 'request_id': 'req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.824374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa48f9d954a4e5296eee8e8af1dbfb0', 'request_id': 'req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.834859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] The token cache is being invalidate because project 9fa48f9d954a4e5296eee8e8af1dbfb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:56:59.835161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.835888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.837459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1081/2199] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:56:59 2022] DELETE /identity/v3/projects/9fa48f9d954a4e5296eee8e8af1dbfb0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:56:59.841565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:56:59.843087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:56:59.856454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603252f-266c-46b7-8cd4-f3d641f7522c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:56:59.925841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:56:59.926759 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:56:59.927060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:56:59.927347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] PATH_INFO: `/v3/users/7596fb15151e481e9e324d13f3a21cd3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:56:59.928677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7596fb15151e481e9e324d13f3a21cd3)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:56:59.930597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:56:59.979322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7596fb15151e481e9e324d13f3a21cd3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.991870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7596fb15151e481e9e324d13f3a21cd3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.992622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7596fb15151e481e9e324d13f3a21cd3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:56:59.993297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7596fb15151e481e9e324d13f3a21cd3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.000145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8ac18155-063b-4853-8f2b-c353c73cde8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7596fb15151e481e9e324d13f3a21cd3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.014805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1119/2200] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:56:59 2022] DELETE /identity/v3/users/7596fb15151e481e9e324d13f3a21cd3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:57:00.199580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.201080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:00.214400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e70a7d-477a-43f8-8e6e-23eea0fbb057 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.274428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.275214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.275494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.275764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] PATH_INFO: `/v3/projects/77f3dc9200394ff1a5f17b523ff9537f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.276994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f3dc9200394ff1a5f17b523ff9537f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.278907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.338424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f3dc9200394ff1a5f17b523ff9537f', 'request_id': 'req-f60b66f4-7182-4940-89fe-7645f219794d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.346650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f3dc9200394ff1a5f17b523ff9537f', 'request_id': 'req-f60b66f4-7182-4940-89fe-7645f219794d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.356927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f3dc9200394ff1a5f17b523ff9537f', 'request_id': 'req-f60b66f4-7182-4940-89fe-7645f219794d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.368654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] The token cache is being invalidate because project 77f3dc9200394ff1a5f17b523ff9537f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:57:00.368937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.369635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f60b66f4-7182-4940-89fe-7645f219794d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:00.371275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1082/2201] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:57:00 2022] DELETE /identity/v3/projects/77f3dc9200394ff1a5f17b523ff9537f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:57:00.377708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.378190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.378335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.402700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:57:00.403904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:00.470623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1120/2202] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:57:00.483366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.483709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.484025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.503009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:57:00.504196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:00.512643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1083/2203] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:57:00.517295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.527979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf204-0f2f-4c8e-9984-18e536c95ba6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.533036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531923-076a-4fcc-b475-114b734577af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.533710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531923-076a-4fcc-b475-114b734577af admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.533986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531923-076a-4fcc-b475-114b734577af admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.534255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531923-076a-4fcc-b475-114b734577af admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.535003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531923-076a-4fcc-b475-114b734577af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 00:57:00.535288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531923-076a-4fcc-b475-114b734577af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.536916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531923-076a-4fcc-b475-114b734577af admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.549205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1121/2204] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 00:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 00:57:00.553503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.563404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7bbe6-1f1f-4f1f-8665-b5129c1916dd None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.570724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.571399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.571676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.571943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.572673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.574604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.603920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1084/2205] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 00:57:00 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 00:57:00.608743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531923-076a-4fcc-b475-114b734577af admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.619455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531923-076a-4fcc-b475-114b734577af admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.678883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.679569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.680128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.680433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.681211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.683420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.706966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1122/2206] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 00:57:00 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 00:57:00.711212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.722709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358071ba-febf-4fc4-bfa0-e99b67977645 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.730149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.730814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.731094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.731369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.732140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.735588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.746041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1085/2207] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:57:00.750756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.760426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c5c2-134f-444a-b931-f76866594210 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.766130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.766788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.767082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.767345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.768092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.769742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea8ded-fefa-4dcf-8680-98d6128d58a9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.779771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1123/2208] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 00:57:00.784107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.796741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407e12-822d-4625-a2d8-51e6d49f1675 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:00.802678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:00.803556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.803861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.804137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] PATH_INFO: `/v3/projects/bba03afc859646358a74ae870a1e31ca/users/91acbc969a1c4ef783ce78d1d2901497/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.819801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba03afc859646358a74ae870a1e31ca, user_id=91acbc969a1c4ef783ce78d1d2901497, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:00.822527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:00.838443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1086/2209] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 00:57:00 2022] PUT /identity/v3/projects/bba03afc859646358a74ae870a1e31ca/users/91acbc969a1c4ef783ce78d1d2901497/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:57:00.849232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45931025-3a00-4ebf-8ac3-7f19a3069d17 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:00.849551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45931025-3a00-4ebf-8ac3-7f19a3069d17 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:00.849840 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45931025-3a00-4ebf-8ac3-7f19a3069d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:00.886541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-45931025-3a00-4ebf-8ac3-7f19a3069d17 None None] MFA Rules not processed for user `91acbc969a1c4ef783ce78d1d2901497`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:57:00.887749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-45931025-3a00-4ebf-8ac3-7f19a3069d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:00.956904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1124/2210] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:57:00.967899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:00.970072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:01.039772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f279516-54a4-4d19-a71e-11367a2c8a6d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:01.043778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:01.044464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:01.044743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:01.045012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:01.054076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:01.055525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:01.064736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1087/2211] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:57:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:57:04.314724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f0e-658e-4f39-a4b3-7619f090e72b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:04.315267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f0e-658e-4f39-a4b3-7619f090e72b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:04.315267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f0e-658e-4f39-a4b3-7619f090e72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:04.338691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-42290f0e-658e-4f39-a4b3-7619f090e72b None None] MFA Rules not processed for user `2b6a0a26494e4e619d504f10ef20fb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:57:04.339905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-42290f0e-658e-4f39-a4b3-7619f090e72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:04.397240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1125/2212] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:57:12.676593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:12.761183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25d549-b9c8-4cc2-b65a-0d83c04beee8 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:12.767034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:12.768119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:12.768446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:12.768723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] PATH_INFO: `/v3/users/91acbc969a1c4ef783ce78d1d2901497` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:12.770148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91acbc969a1c4ef783ce78d1d2901497)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:12.772145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:12.829436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91acbc969a1c4ef783ce78d1d2901497'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:12.841975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91acbc969a1c4ef783ce78d1d2901497'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:12.849169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91acbc969a1c4ef783ce78d1d2901497'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:12.849928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91acbc969a1c4ef783ce78d1d2901497'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:12.858002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91acbc969a1c4ef783ce78d1d2901497'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:12.865480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1088/2213] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 00:57:12 2022] DELETE /identity/v3/users/91acbc969a1c4ef783ce78d1d2901497 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:57:12.870342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:12.870846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:12.870973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:12.893121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:57:12.894349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:12.958614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1126/2214] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:57:12.968655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:12.972147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:13.039557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b0a27-e7f8-4c94-827a-40e59d290d06 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:13.044791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:13.045450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:13.045704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:13.045976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:13.056636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:13.058087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:13.069355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1089/2215] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:57:13.415287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:13.417554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:13.431333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed0a74-d5ef-43c1-97b3-d50bd783e187 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:13.438335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:13.439364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:13.439649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:13.440110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] PATH_INFO: `/v3/projects/bba03afc859646358a74ae870a1e31ca` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:13.441443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba03afc859646358a74ae870a1e31ca)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:13.443646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:13.500140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba03afc859646358a74ae870a1e31ca', 'request_id': 'req-fe417960-8a65-41d0-ac47-9d3fbbbbd809'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:13.506604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba03afc859646358a74ae870a1e31ca', 'request_id': 'req-fe417960-8a65-41d0-ac47-9d3fbbbbd809'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:13.516240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba03afc859646358a74ae870a1e31ca', 'request_id': 'req-fe417960-8a65-41d0-ac47-9d3fbbbbd809'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:13.528162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] The token cache is being invalidate because project bba03afc859646358a74ae870a1e31ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 00:57:13.528457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:13.529210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 00:57:13.531202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1127/2216] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 00:57:13 2022] DELETE /identity/v3/projects/bba03afc859646358a74ae870a1e31ca => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 00:57:56.892849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:57:56.894473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:56.905818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c06691-1997-4dbe-8d53-08f6adee4a23 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:57:56.962491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:57:56.963143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:57:56.963421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:57:56.963696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:57:56.965763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:57:57.060739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:57:57.062197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:57:57.082740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1090/2217] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:57:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:10.537487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:10.540202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:10.600326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe417960-8a65-41d0-ac47-9d3fbbbbd809 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:10.605063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:10.605732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:10.606017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:10.606988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:10.608661 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:10.670797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:10.672269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:10.681361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1128/2218] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:17.204279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:17.206089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:17.221732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce141d8-9f2d-484a-b784-77585cbca800 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:17.230013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:17.230652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:17.230937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:17.231578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:17.233659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:17.294616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:17.296225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:17.305967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1091/2219] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:18.636548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:18.645859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0d26-f198-42a7-99c0-a5325ed9921f service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:18.650380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:18.652012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:18.652012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:18.652012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:18.654228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:18.723956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:18.726147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:18.737272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1129/2220] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:27.375502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:27.377066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:27.391372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e1254-5d94-47f8-abea-06a1d3e66ae5 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:27.395958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:27.396793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:27.397069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:27.397340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:27.399488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:27.409758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:27.411275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26032eca-fc35-4118-8fca-f0f3cebe6935 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:27.419792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1092/2221] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:28.902792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:28.916025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4aa6e3-f415-454c-8d3a-5313ed594acf service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:28.922962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:28.923605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:28.923885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:28.924158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:28.926192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:28.984958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:28.986536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:28.995484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1130/2222] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:33.060873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:33.061389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:33.061501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:33.079831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] MFA Rules not processed for user `6a41fc2b6e1c4032bfa408eaf36887ee`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:58:33.081121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:33.089012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1093/2223] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 00:58:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:58:33.411253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:33.422447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0ec33-f3c6-4444-af66-7545d1407146 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:33.427298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:33.427939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:33.428223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:33.428540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:33.430764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:33.441444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:33.442853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:33.451508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1131/2224] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:33.797320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:33.798755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:33.875929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da3ba-8f41-4ec0-af77-b98783aefef2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:33.882280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:33.883079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:33.883600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:33.884098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:33.886065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:33.898820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:33.900815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:33.923341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1094/2225] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:35.859799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:35.880333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b5aa7-ce35-4360-babe-a782498866a6 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:35.891442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:35.892503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:35.892989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:35.893533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:35.896834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:35.913080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:35.914689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767993c5-7e3c-4d8b-bf1d-b9918bae2a2f service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:35.929602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1132/2226] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:36.925444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:36.940985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fabb1b-cd55-42de-a93b-be6af3672ea5 service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:36.947895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:36.948610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:36.948892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:36.949163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:36.951212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:36.964596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:36.966448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:36.979447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1095/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:37.608606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca07b5e-8ca4-41dc-8a9b-5fc9e64aefc9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:37.609081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca07b5e-8ca4-41dc-8a9b-5fc9e64aefc9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:37.609168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca07b5e-8ca4-41dc-8a9b-5fc9e64aefc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:37.665130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-fca07b5e-8ca4-41dc-8a9b-5fc9e64aefc9 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:58:37.666436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fca07b5e-8ca4-41dc-8a9b-5fc9e64aefc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:37.735149 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1133/2228] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 00:58:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:58:37.781396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:37.782905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:37.869919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da7c2-dc10-47f1-b37c-0a4b2f6e0492 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:37.875115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:37.875758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:37.876096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:37.876359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:37.884773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:37.886355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf6209-ce32-4ace-8924-7c12a207165d service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:37.894501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1096/2229] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 00:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 00:58:52.802221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:52.802851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:52.802851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:52.804238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1134/2230] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 00:58:52.810607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162874-5ee2-4acc-b314-4759cf117115 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:52.811131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162874-5ee2-4acc-b314-4759cf117115 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:52.811131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162874-5ee2-4acc-b314-4759cf117115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:52.850431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-01162874-5ee2-4acc-b314-4759cf117115 None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:58:52.851644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-01162874-5ee2-4acc-b314-4759cf117115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:52.863142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1097/2231] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:58:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:58:52.875713 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:52.877698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:52.958581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47fc7-01b3-4984-8c15-cf2a7e6e27ae None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:52.963548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:52.964196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:52.964500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:52.964783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:52.982463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:52.983919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2aa69-a19a-4ce3-9d41-fbf9a8dca755 service glance] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:52.995226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1135/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:53.070032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.070542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.070938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] PATH_INFO: `/` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.072685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1098/2233] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:58:53.077814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0902-3e46-4fae-b1d4-5722aaf9b46c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.078130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0902-3e46-4fae-b1d4-5722aaf9b46c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.078370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0902-3e46-4fae-b1d4-5722aaf9b46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.120022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4bde0902-3e46-4fae-b1d4-5722aaf9b46c None None] MFA Rules not processed for user `692dbeda127948b3812e6153241e7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:58:53.121230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4bde0902-3e46-4fae-b1d4-5722aaf9b46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:53.128166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1136/2234] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:58:53.139966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:53.141711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:53.156953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50a266-2b93-405c-978b-ae01d32b1d4d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:53.161889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:53.162528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.162806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.163087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.171654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:53.173894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fd842-06f7-4d13-87d3-6f653a1f3905 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:53.182814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1099/2235] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:58:53.276633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7b9-dbbf-4f18-a7a4-b9e7bda4ba71 None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.276922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7b9-dbbf-4f18-a7a4-b9e7bda4ba71 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.277200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7b9-dbbf-4f18-a7a4-b9e7bda4ba71 None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.278596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1137/2236] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 12 00:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 00:58:53.286152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.286439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.286718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.307664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] MFA Rules not processed for user `736e16d1bc05411faea9c943f930acb1`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 00:58:53.308890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:58:53.320183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1100/2237] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 12 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 00:58:53.331558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ca7b9-dbbf-4f18-a7a4-b9e7bda4ba71 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:58:53.347934 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ca7b9-dbbf-4f18-a7a4-b9e7bda4ba71 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:58:53.355754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:58:53.356726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:58:53.357156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:58:53.357482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:58:53.370495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:58:53.371864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:58:53.383039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1138/2238] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:58:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 00:59:48.287950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 00:59:48.289529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:59:48.301690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e207b-3ad8-4ab3-9421-c1270380800a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 00:59:48.306020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 00:59:48.306648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 00:59:48.306928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 00:59:48.307200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 00:59:48.309276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 00:59:48.383417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 00:59:48.384901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 00:59:48.394582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1101/2239] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 00:59:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:00:13.089058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:00:13.100504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904dd7-9a76-4632-b31a-1822115c253c service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:00:13.107630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:00:13.108305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:00:13.108587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:00:13.108862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:00:13.110802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:00:13.175327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:00:13.177487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:00:13.186580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1139/2240] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:00:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:02:04.807108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:02:04.819655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c1aef-f251-44ba-ad99-444f41cf234f service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:02:04.825833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:02:04.826467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:02:04.826748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:02:04.827024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:02:04.828927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:02:04.842205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:02:04.843787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:02:04.854828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1102/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:02:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:02:57.144579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:02:57.154728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60608-9d12-46e4-8b11-89cfe2603dd4 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:02:57.159550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:02:57.160384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:02:57.160663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:02:57.160963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:02:57.172730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:02:57.174161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:02:57.185584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1140/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:02:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:10.546825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:10.562073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055938c-32c2-4d59-aa24-87f207916344 service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:10.568506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:10.569280 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:10.569563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:10.569837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:10.579862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:10.581335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:10.611271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1103/2243] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:17.388630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:17.400030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8116a-e6d3-4c23-88f2-2b3d6fcdbfe4 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:17.407214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:17.407850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:17.408132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:17.408446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:17.420414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:17.421990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:17.450839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1141/2244] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:18.851425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:18.864271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12187383-e7e2-4b50-8ef0-22860a5bb2a9 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:18.870539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:18.871183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:18.871459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:18.871730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:18.879639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:18.881348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a936b5-d474-4e34-bda4-a549c0f7e96d service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:18.909882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1104/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:29.943394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:29.955709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb1c14-888e-42f7-b409-191960883891 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:29.960281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:29.960947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:29.961227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:29.961495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:29.971610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:29.973074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:30.002204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1142/2246] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:38.722087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:38.722620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:38.722620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:38.763886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] MFA Rules not processed for user `7facb556fc714c5fac568a21fa5ffcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:03:38.765113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:03:38.773159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1105/2247] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:03:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:03:39.354072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:39.363963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6963-c0ff-4e4e-8ab8-38f87441c4d3 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:39.368752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:39.369491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:39.369908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:39.370209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:39.377966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:39.379390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:39.387707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1143/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:39.736186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:39.737903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:03:39.753273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b6b91-efd1-4fde-a37e-9e5f2b5a0763 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:39.758537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:39.759186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:39.759466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:39.760160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:39.768741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:39.770330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:39.780268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1106/2249] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:41.586449 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:41.596486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2d72e-949f-4d17-9d2f-122e95fa1fef service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:41.601279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:41.601942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:41.602224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:41.602536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:41.612295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:41.613720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:41.622926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1144/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:42.711890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:42.724479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456a81-b775-4081-91fc-f8ae877ecc9b service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:42.731785 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:42.732480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:42.732764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:42.733101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:42.742689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:42.744411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:42.755387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1107/2251] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:03:47.428750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:03:47.442208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041222b-01cc-4675-b09a-5a0d1023b81a service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:03:47.447833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:03:47.448766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:03:47.448903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:03:47.449360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:03:47.458592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:03:47.460169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d8294-bf29-4761-af80-40aea2eda31a service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:03:47.470567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1145/2252] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:03:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:04:48.650803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:04:48.663586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1a1f2-6ce8-4733-8566-6956acd426fe service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:04:48.668084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:04:48.668735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:48.669015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:48.669283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:48.678959 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:04:48.680414 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:04:48.708552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1108/2253] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:04:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:04:52.908235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9370b3-b738-4447-9606-d7732db89be4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:52.908770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9370b3-b738-4447-9606-d7732db89be4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:52.908770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9370b3-b738-4447-9606-d7732db89be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:52.927526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9d9370b3-b738-4447-9606-d7732db89be4 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:04:52.928727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9d9370b3-b738-4447-9606-d7732db89be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:04:52.937570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1146/2254] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 12 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:04:52.986920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:04:52.999559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfece7-65b8-4589-89d0-1153884f685d service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:04:53.005482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:04:53.006125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:53.006429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:53.006707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:53.014856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:04:53.016471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bb44a-3d41-402b-8163-e286a3094bf5 service swift] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:04:53.024231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1109/2255] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 01:04:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 01:04:54.605024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69a80-b535-456c-8e28-d29902acd21a None None] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:54.605641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69a80-b535-456c-8e28-d29902acd21a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:54.605641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69a80-b535-456c-8e28-d29902acd21a None None] PATH_INFO: `/` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:54.606924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1147/2256] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 12 01:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 01:04:54.612507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781669-ee7f-4da4-b18b-e3732d5180b1 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:54.612963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781669-ee7f-4da4-b18b-e3732d5180b1 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:54.613079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781669-ee7f-4da4-b18b-e3732d5180b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:54.651560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-14781669-ee7f-4da4-b18b-e3732d5180b1 None None] MFA Rules not processed for user `b7dc2677d5c74bc7ad7770c3575f2b74`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:04:54.653990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-14781669-ee7f-4da4-b18b-e3732d5180b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:04:54.663432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1110/2257] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:04:54.688678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d69a80-b535-456c-8e28-d29902acd21a None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:04:54.705289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d69a80-b535-456c-8e28-d29902acd21a None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:04:54.712445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:04:54.713559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:54.713985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:54.714411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:54.725295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:04:54.727356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:04:54.739005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1148/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:04:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:04:57.170441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:04:57.170969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:04:57.170969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:04:57.210863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] MFA Rules not processed for user `a71d4761b81447a194e2141442ceef1d`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:04:57.212065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:04:57.219124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1111/2259] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:04:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:05:12.885163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:05:12.928943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506014ab-ac08-435e-a0a5-f70e9ba0bd49 service placement] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:05:12.935054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:05:12.935700 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:05:12.935993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:05:12.936292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:05:12.944482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:05:12.946065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:05:12.974737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1149/2260] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:05:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:07:10.796344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:07:10.843069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b804d2b-f176-4330-b770-64e2a9709ee5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:07:10.850833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:07:10.851925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:07:10.852210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:07:10.852655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:07:10.866738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:07:10.869184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:07:10.882377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1112/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:07:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:08:10.445293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:08:10.465756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db83a4-7aae-4db2-b2f6-3bec4b4ff3c9 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:08:10.470860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:08:10.471920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:08:10.472204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:08:10.472523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:08:10.482913 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:08:10.484490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:08:10.493512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1150/2262] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:08:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:08:17.954155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:08:17.955794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:08:18.048356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2088fc-7e83-477c-962c-2fb9b31cafeb service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:08:18.053351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:08:18.054030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:08:18.054311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:08:18.054583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:08:18.056848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:08:18.121690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:08:18.123100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:08:18.132376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1113/2263] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:08:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:08:18.551361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:08:18.608031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fd69c-e8c4-43c3-9e62-be960652acbc service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:08:18.612299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:08:18.612967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:08:18.613250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:08:18.613525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:08:18.622779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:08:18.624223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:08:18.633394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1151/2264] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:08:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:08:29.590777 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:08:29.599847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223c06-0f78-4cdc-9766-a6a0dee80905 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:08:29.604856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:08:29.605486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:08:29.605763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:08:29.606036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:08:29.607911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:08:29.682643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:08:29.684024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:08:29.694496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1114/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:08:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:09:18.462836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:09:18.464574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:09:18.542265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae165e-7203-4a65-a1e7-d6e8cf49dd8a service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:09:18.546521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:09:18.547164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:09:18.547445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:09:18.547709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:09:18.549941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:09:18.560772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:09:18.562190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:09:18.570937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1152/2266] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:09:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:09:30.219006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:09:30.229525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5406-934d-459b-803f-6869e584d1b5 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:09:30.233387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:09:30.234028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:09:30.234307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:09:30.234583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:09:30.236712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:09:30.340382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:09:30.341833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:09:30.369024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1115/2267] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:09:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:09:48.261460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:09:48.264232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:09:48.287270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3572ee-823a-447d-ba00-99b32fe7a8fc service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:09:48.294233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:09:48.294868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:09:48.295145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:09:48.295415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:09:48.303434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:09:48.305031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:09:48.313724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1153/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:09:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:10:12.375535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:10:12.378273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:10:12.395601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbb495-e892-4944-87df-52a593ed5663 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:10:12.400201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:10:12.400889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:10:12.401194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:10:12.401463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:10:12.411085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:10:12.412485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:10:12.422043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1116/2269] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:10:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:11:08.426730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:11:08.441417 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280cf24-0da3-46d8-b620-99b4e456fc63 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:11:08.453496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:11:08.454170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:11:08.454641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:11:08.454919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:11:08.457094 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:11:08.472819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:11:08.477595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:11:08.493330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1154/2270] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:12:16.750647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:12:16.827443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f00c4-e49f-4d29-87ca-0008f97e168b service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:12:16.832567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:12:16.833218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:12:16.833493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:12:16.833766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:12:16.835823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:12:16.863136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:12:16.865127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:12:16.878491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1117/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:12:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:13:18.721629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:13:18.733705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dae4-690f-4d97-960e-e1db7493b404 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:13:18.740866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:13:18.741543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:13:18.741823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:13:18.742093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:13:18.744569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:13:18.816284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:13:18.817881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:13:18.826894 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1155/2272] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:13:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:20.063384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:20.073473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f1593-5685-41e2-901e-bacc7ca7dc3f service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:20.078059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:20.078709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:20.078990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:20.079259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:20.087133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:20.088622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:20.097101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1118/2273] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:30.375167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:30.385838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c240943-c5fd-4ce2-9f6f-58c5f293d296 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:30.391207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:30.391843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:30.392122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:30.392422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:30.405481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:30.406835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:30.418469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1156/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:32.758397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:32.768458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7450e1-71ce-4e1f-b3c9-93ba86b77a70 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:32.773224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:32.773899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:32.774204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:32.774480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:32.784425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:32.785862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:32.814102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1119/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:32.859246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:32.868071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaff64b-681c-439a-993b-14bda5fdfd33 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:32.871696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:32.872364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:32.872641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:32.872906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:32.881557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:32.882985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:32.909942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1157/2276] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:32.957572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:32.966735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc2803-b631-4ed0-aa8d-e9aa1ae9c5a2 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:32.971203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:32.971847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:32.972128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:32.972438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:32.974359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:14:33.055220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:33.056700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:33.080404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1120/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:14:48.269600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:14:48.285779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d049d-878b-4047-bd90-8eaef7a34f9c service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:14:48.293100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:14:48.293749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:14:48.294027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:14:48.294295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:14:48.296855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:14:48.371707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:14:48.373852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:14:48.382844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1158/2278] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:15:12.109396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:15:12.122480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c1de-08aa-43ad-8897-fd8511c9b9ab service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:15:12.129644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:15:12.130293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:15:12.130552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:15:12.130829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:15:12.132797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:15:12.202436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:15:12.203936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:15:12.212992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1121/2279] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:15:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:16:35.444335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:35.469739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f944c6-0095-4092-9f60-68d2b9e9798f service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:35.475861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:35.476540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:35.476825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:35.477102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:35.485950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:35.488013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:35.498407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1159/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:16:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:16:39.131038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:39.149269 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2709e9-a692-4323-b072-e9061e57952d service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:39.153027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:39.153676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:39.153954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:39.154221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:39.156142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:39.248833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:39.250275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:39.274132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1122/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:16:43.068723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.070747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:43.085515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bccdb5-9471-4809-8f05-9081da2e2c14 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.090589 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.091472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.091753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.092026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] PATH_INFO: `/v3/users/5db47a076f5d4650b0a7a6945ae70a52` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.093569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db47a076f5d4650b0a7a6945ae70a52)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.095497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.142823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db47a076f5d4650b0a7a6945ae70a52'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.156114 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db47a076f5d4650b0a7a6945ae70a52'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.156949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db47a076f5d4650b0a7a6945ae70a52'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.157692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db47a076f5d4650b0a7a6945ae70a52'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.165777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-79fc97b5-50a1-4cf6-a24f-8c4f29201634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db47a076f5d4650b0a7a6945ae70a52'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.181868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1160/2282] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:16:43 2022] DELETE /identity/v3/users/5db47a076f5d4650b0a7a6945ae70a52 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:16:43.375039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.384892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aaf9c-e0a8-49be-91ac-2776c86e6853 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.447898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.448812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.449118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.449362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] PATH_INFO: `/v3/projects/f6e1985d38344304bc3b1e8f4db5e896` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.450536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e1985d38344304bc3b1e8f4db5e896)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.452477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.506039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e1985d38344304bc3b1e8f4db5e896', 'request_id': 'req-cbe316f7-2002-4b6f-9061-fbc2685aba8f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.512014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e1985d38344304bc3b1e8f4db5e896', 'request_id': 'req-cbe316f7-2002-4b6f-9061-fbc2685aba8f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.521210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e1985d38344304bc3b1e8f4db5e896', 'request_id': 'req-cbe316f7-2002-4b6f-9061-fbc2685aba8f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.532199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] The token cache is being invalidate because project f6e1985d38344304bc3b1e8f4db5e896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:16:43.532512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.533203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-cbe316f7-2002-4b6f-9061-fbc2685aba8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:16:43.534977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1123/2283] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:16:43 2022] DELETE /identity/v3/projects/f6e1985d38344304bc3b1e8f4db5e896 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:16:43.541466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.541751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.542028 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.601971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:16:43.603244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:43.689283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1161/2284] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:16:43.697723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.698186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.698529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.718617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:16:43.719836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:43.732884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1124/2285] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:16:43.733757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.743729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d3c74-673a-4fb5-b332-60f914ba97f8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.748366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.749039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.749321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.749610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.750347 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:16:43.750659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.752299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.768071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1162/2286] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:16:43.770949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.780597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eda90f-a324-4445-9584-42c4b6bb29aa None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.785508 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.786247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.786597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.787455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.787794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.791428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.828735 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1125/2287] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:16:43 2022] POST /identity/v3/projects => generated 357 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:16:43.833229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.844134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02d0a-2936-41ca-a06a-ba09f18ffff5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.902895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.903513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.903803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.904081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.904834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.906799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.927250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1163/2288] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:16:43 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:16:43.931496 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.941855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8530da-dda5-40d8-b218-f7792d2e50bd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.946353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.947008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.947285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.947648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.948300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.949909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:43.960399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1126/2289] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:16:43.965157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:43.980732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbd090-d92b-4b89-b77f-afb7a4779fe9 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:43.988597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:43.989262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:43.989546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:43.989820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:43.990580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:43.992175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c6bd4-8a99-42ed-9aad-581a447136cb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.003312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1164/2290] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:16:44.009654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.022662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf81428-ed40-4fcf-a2ea-041ea3c30325 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.028880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.029775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.030051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.030321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] PATH_INFO: `/v3/projects/4daeeba5e3ed400eb1351b9fa8f3a612/users/7b971f10cacf40c3b7c5bb2045d5c9ea/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.046776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4daeeba5e3ed400eb1351b9fa8f3a612, user_id=7b971f10cacf40c3b7c5bb2045d5c9ea, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.049503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.065396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1127/2291] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:16:44 2022] PUT /identity/v3/projects/4daeeba5e3ed400eb1351b9fa8f3a612/users/7b971f10cacf40c3b7c5bb2045d5c9ea/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:16:44.076000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.076596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.076596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.103966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] MFA Rules not processed for user `7b971f10cacf40c3b7c5bb2045d5c9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:16:44.105192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:44.171064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1165/2292] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:16:44.183529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.257615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b2b10-46c2-4a2a-bb13-1016d75bb08a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.263329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.264026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.264334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.264613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.265298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.267212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.296157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1128/2293] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:16:44 2022] POST /identity/v3/projects => generated 357 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:16:44.302465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.318539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a82de-862b-476f-82f3-b8e41da5ab23 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.386384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.387033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.387356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.387696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.388383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.390551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.411548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1166/2294] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:16:44 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:16:44.415624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.426236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44933e3-e9e9-49f1-8926-776ac5e3a7a3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.431260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.431905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.432184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.432526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.433224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.434819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.445404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1129/2295] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:16:44.449716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.460771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c395b-e03d-44f3-8ee2-d6d7d3275500 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.466635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.467515 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.467764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.468059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] PATH_INFO: `/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7/users/21e45af3398e47cbb897325e4044edfd/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.483488 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5859cbc122da4e07a3ec8f3686db2ad7, user_id=21e45af3398e47cbb897325e4044edfd, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.486215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.502029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1167/2296] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:16:44 2022] PUT /identity/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7/users/21e45af3398e47cbb897325e4044edfd/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:16:44.506289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.520990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535b71-8d5b-478f-aaa1-c2e489db8944 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.582513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.583192 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.583480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.583763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.584562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.586197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5783a3-82cb-47e2-ade1-7a690bef8d6b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.596471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1130/2297] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:16:44.600838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:16:44.611568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723969c5-34a5-4eb1-a7f7-e4fe2ae848c3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:16:44.616873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:16:44.617757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.618033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.618304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] PATH_INFO: `/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7/users/21e45af3398e47cbb897325e4044edfd/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.620412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5859cbc122da4e07a3ec8f3686db2ad7, user_id=21e45af3398e47cbb897325e4044edfd, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:16:44.623041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:16:44.633878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1168/2298] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:16:44 2022] PUT /identity/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7/users/21e45af3398e47cbb897325e4044edfd/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:16:44.649218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29a65f-601d-4e36-a57d-14aefd2185ef None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:16:44.649537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29a65f-601d-4e36-a57d-14aefd2185ef None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:16:44.649800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29a65f-601d-4e36-a57d-14aefd2185ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:16:44.679851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9f29a65f-601d-4e36-a57d-14aefd2185ef None None] MFA Rules not processed for user `21e45af3398e47cbb897325e4044edfd`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:16:44.681173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9f29a65f-601d-4e36-a57d-14aefd2185ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:16:44.755202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1131/2299] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:17:18.698884 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:17:18.701905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:17:18.813558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd178ab6-3846-4403-a1bd-51eed2775782 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:17:18.819624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:17:18.820385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:17:18.820671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:17:18.820947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:17:18.822854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:17:18.834420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:17:18.836055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:17:18.846345 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1169/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:17:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:10.345227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:10.345778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:10.345778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:10.406268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:18:10.407465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:10.488523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1132/2301] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 12 01:18:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:18:10.537443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:10.538896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:10.647917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2517abc-7899-4668-a342-cd0bd06eea60 service placement] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:10.653966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:10.654624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:10.654887 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:10.655182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:10.663829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:10.665395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:10.673307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1170/2302] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 01:18:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 01:18:11.262901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:11.264420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:11.319766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c843c-22b9-4000-8a57-14791e09e90f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:11.323456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:11.324102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:11.324424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:11.324697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:11.326422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:11.386998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:11.388460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:11.398525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1133/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:18.492612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:18.494058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:18.518836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee97db9-a86c-43b1-b32f-ea4758924eaa service swift] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:18.524019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:18.524936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:18.525216 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:18.525492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] PATH_INFO: `/v3/users/9e29b068b9b34827be27bb4f83bae523` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:18.526777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e29b068b9b34827be27bb4f83bae523)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:18.528763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:18.576307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e29b068b9b34827be27bb4f83bae523'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.588001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e29b068b9b34827be27bb4f83bae523'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.588676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e29b068b9b34827be27bb4f83bae523'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.589270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e29b068b9b34827be27bb4f83bae523'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.595762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e29b068b9b34827be27bb4f83bae523'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.609948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1171/2304] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:18:18 2022] DELETE /identity/v3/users/9e29b068b9b34827be27bb4f83bae523 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:18:18.623421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:18.625598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:18.715404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bae25-3ff6-4c50-a171-60867f8a0853 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:18.780764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:18.781388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:18.781770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:18.782257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:18.784468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:18.792883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:18.794288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:18.856162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:18.857952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac68b-189d-466b-b622-1ae357424236 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:18.858089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:18.862370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:18.863236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:18.863513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:18.863794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] PATH_INFO: `/v3/projects/a00cd6b85252457abfea8cdc68dbbebe` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:18.874264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00cd6b85252457abfea8cdc68dbbebe)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:18.876156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:18.876633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1134/2305] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:18.926130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00cd6b85252457abfea8cdc68dbbebe', 'request_id': 'req-fdd6727a-0129-4363-8da4-c78e2a14ff3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.933067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00cd6b85252457abfea8cdc68dbbebe', 'request_id': 'req-fdd6727a-0129-4363-8da4-c78e2a14ff3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.943259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00cd6b85252457abfea8cdc68dbbebe', 'request_id': 'req-fdd6727a-0129-4363-8da4-c78e2a14ff3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.954536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] The token cache is being invalidate because project a00cd6b85252457abfea8cdc68dbbebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:18:18.954842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.955476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:18.957062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1172/2306] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:18:18 2022] DELETE /identity/v3/projects/a00cd6b85252457abfea8cdc68dbbebe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:18:18.961667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:18.972224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa35ef-d904-4fdf-b594-288f14e0814d service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:19.032043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:19.032900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:19.033179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:19.033448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] PATH_INFO: `/v3/users/d6aaee3409e04fe18e90733809be20df` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:19.048212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6aaee3409e04fe18e90733809be20df)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:19.050212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:19.100303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6aaee3409e04fe18e90733809be20df'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.111563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6aaee3409e04fe18e90733809be20df'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.119032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6aaee3409e04fe18e90733809be20df'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.119797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6aaee3409e04fe18e90733809be20df'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.128498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c72f4cfa-b7bd-4187-87e7-8a179247b7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6aaee3409e04fe18e90733809be20df'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.135607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1135/2307] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:18:18 2022] DELETE /identity/v3/users/d6aaee3409e04fe18e90733809be20df => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:18:19.310805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:19.312441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:19.326201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6727a-0129-4363-8da4-c78e2a14ff3f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:19.384121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:19.385063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:19.385336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:19.385605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] PATH_INFO: `/v3/projects/b320f1895ceb4ee0b3478453c108f788` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:19.396592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b320f1895ceb4ee0b3478453c108f788)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:19.398538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:19.446264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b320f1895ceb4ee0b3478453c108f788', 'request_id': 'req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.454468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b320f1895ceb4ee0b3478453c108f788', 'request_id': 'req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.464590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b320f1895ceb4ee0b3478453c108f788', 'request_id': 'req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.475184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] The token cache is being invalidate because project b320f1895ceb4ee0b3478453c108f788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:18:19.475468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.476126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:18:19.477739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1173/2308] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:18:19 2022] DELETE /identity/v3/projects/b320f1895ceb4ee0b3478453c108f788 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:18:27.446328 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:27.446896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:27.446896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:27.474467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] MFA Rules not processed for user `6a41fc2b6e1c4032bfa408eaf36887ee`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:18:27.475677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:27.534010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1136/2309] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:18:27.881610 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:27.884141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:27.942904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a65c0-94db-4fb4-a8d1-0d12e4b64615 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:27.946909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:27.947852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:27.948291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:27.948601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:27.950617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:27.960527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:27.961936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:27.969800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1174/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:28.299532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:28.301253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:28.373018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfb366-b39f-4ed3-9cf5-8e481740ad7f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:28.378437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:28.379067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:28.379343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:28.379614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:28.381421 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:28.392638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:28.394190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:28.404734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1137/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:34.081631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:34.084643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:34.198619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0da67-d2bd-4e6d-8393-72d5b27d5dd4 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:34.210933 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:34.212824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:34.213435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:34.213916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:34.223851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:34.230613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:34.233979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:34.248464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371a097-c39e-4b81-a741-e8dafc424697 service placement] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:34.252763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:34.253420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:34.253707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:34.253979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:34.256231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:34.331155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:34.331155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:34.336776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:34.338551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:34.352369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1175/2312] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:34 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:34.360863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1138/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:34 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:18:36.194680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:18:36.196282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:36.272705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789d253-4b9f-46e6-9c10-f232e5f855f7 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:18:36.282162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:18:36.283206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:18:36.283722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:18:36.284083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:18:36.286380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:18:36.301857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:18:36.304498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:18:36.318836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1176/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:18:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:19:25.466092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:19:25.486086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3337397-9e05-4c37-8262-c90440b95c72 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:19:25.490830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:19:25.491469 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:19:25.491743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:19:25.492013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:19:25.494279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:19:25.518908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:19:25.520413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:19:25.528922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1139/2315] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:19:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:34.454305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:34.457851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718d70-b593-45f3-87b2-d25553056ac9 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:34.462412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:34.463146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:34.463426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:34.463729 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:34.465683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:34.474933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:34.484868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45407df0-2dd7-4d4b-8fc2-d639d952cbd3 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:34.489370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:34.490015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:34.490288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:34.490554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:34.492695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:34.542557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:34.544332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:34.553442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:34.553638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1177/2316] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:34.554995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:34.564678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1140/2317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:34.607598 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:34.616356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238bf3-578d-459c-b0ee-9235d254cb0f service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:34.620099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:34.620797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:34.621075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:34.621373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:34.623281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:34.692854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:34.694162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:34.713977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1178/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:34.955250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:34.964278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef70e-e010-4d2a-bcca-c6335856afc6 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:34.968096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:34.968790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:34.969102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:34.969471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:34.971583 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-67a049ab-8304-49bb-b743-ed26de1d3259 service cinder] Could not find project: a00cd6b85252457abfea8cdc68dbbebe.: keystone.exception.ProjectNotFound: Could not find project: a00cd6b85252457abfea8cdc68dbbebe. May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 12 01:20:35.005277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 01:20:35.006597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._validate_token_resources() May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 01:20:35.007894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 01:20:35.009152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 01:20:35.010293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a00cd6b85252457abfea8cdc68dbbebe. May 12 01:20:35.010293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 01:20:35.010293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1141/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:34 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 12 01:20:35.176642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:35.178120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:35.246503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cfafe-56b7-4301-9ca4-7e1fb408288b service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:35.251100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:35.252035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:35.252339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:35.252617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] PATH_INFO: `/v3/users/7b971f10cacf40c3b7c5bb2045d5c9ea` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:35.253815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b971f10cacf40c3b7c5bb2045d5c9ea)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:35.255745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:35.301233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b971f10cacf40c3b7c5bb2045d5c9ea'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.314835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b971f10cacf40c3b7c5bb2045d5c9ea'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.315498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b971f10cacf40c3b7c5bb2045d5c9ea'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.316137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b971f10cacf40c3b7c5bb2045d5c9ea'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.322870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b971f10cacf40c3b7c5bb2045d5c9ea'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.337647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1179/2320] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:20:35 2022] DELETE /identity/v3/users/7b971f10cacf40c3b7c5bb2045d5c9ea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:35.342209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:35.342580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:35.342918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:35.361211 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:35.362467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:35.431060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1142/2321] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:35.442832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:35.444620 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:35.528361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccb429-b4cb-4082-aac8-31e9256acf9b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:35.532499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:35.533169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:35.533451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:35.533775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:35.542878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:35.544344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:35.554200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1180/2322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:35.851182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:35.853063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:35.865763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826a7e3-d09f-4a70-abbf-42eb4ad3cbaf None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:35.871151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:35.872016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:35.872315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:35.872641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] PATH_INFO: `/v3/projects/4daeeba5e3ed400eb1351b9fa8f3a612` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:35.873815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4daeeba5e3ed400eb1351b9fa8f3a612)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:35.875744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:35.935467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4daeeba5e3ed400eb1351b9fa8f3a612', 'request_id': 'req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.944518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4daeeba5e3ed400eb1351b9fa8f3a612', 'request_id': 'req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.953053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4daeeba5e3ed400eb1351b9fa8f3a612', 'request_id': 'req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.966146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] The token cache is being invalidate because project 4daeeba5e3ed400eb1351b9fa8f3a612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:20:35.966429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.967103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:35.968747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1143/2323] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:20:35 2022] DELETE /identity/v3/projects/4daeeba5e3ed400eb1351b9fa8f3a612 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:20:35.974223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:35.991373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582aa9f-84d3-4ebe-969e-432bd563e311 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.057283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.058528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.058969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.059350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] PATH_INFO: `/v3/users/21e45af3398e47cbb897325e4044edfd` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.061426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e45af3398e47cbb897325e4044edfd)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:36.064596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:36.129342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e45af3398e47cbb897325e4044edfd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.144801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e45af3398e47cbb897325e4044edfd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.145892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e45af3398e47cbb897325e4044edfd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.146334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e45af3398e47cbb897325e4044edfd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.156485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-d1bc20a6-8c2f-4206-ab71-506b94bb4e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e45af3398e47cbb897325e4044edfd'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.174950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1181/2324] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:20:35 2022] DELETE /identity/v3/users/21e45af3398e47cbb897325e4044edfd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:36.487474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:36.489412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:36.503973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c94a3a-25b6-4d1d-85fa-51ba42bfa4d1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.574515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.575309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.575602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.575847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] PATH_INFO: `/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.577138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5859cbc122da4e07a3ec8f3686db2ad7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:36.579103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:36.632113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5859cbc122da4e07a3ec8f3686db2ad7', 'request_id': 'req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.639161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5859cbc122da4e07a3ec8f3686db2ad7', 'request_id': 'req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.648178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5859cbc122da4e07a3ec8f3686db2ad7', 'request_id': 'req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.658575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] The token cache is being invalidate because project 5859cbc122da4e07a3ec8f3686db2ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:20:36.658575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.659244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0ffcd3-2836-471d-9b3e-0fab0ddede46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:36.660847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1144/2325] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:20:36 2022] DELETE /identity/v3/projects/5859cbc122da4e07a3ec8f3686db2ad7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:36.665861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.666381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.666499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.684463 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:36.685673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:36.751289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1182/2326] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:36.760096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.760423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.760706 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.780207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:36.781569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:36.790830 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1145/2327] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:36.795467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:36.805969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d9976-5c4e-40d3-9eb2-06a67f2c8409 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.812635 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.813294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.813580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.813861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.814789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:20:36.815096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:36.816768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:36.829509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1183/2328] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:20:36.834276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:36.844238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2de1e4-0817-46d9-b5f0-53ff6d7741a3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.850066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.850740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.851021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.851295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.852012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:36.853956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:36.882405 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1146/2329] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:20:36 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:36.888581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:36.899466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2959f-b53c-464c-98da-867f91c133fb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.952566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.953231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.953508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.953781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.954460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:36.956584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:36.978160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1184/2330] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:20:36 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:36.982385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:36.992499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca76df-51a6-4d81-9a39-5fab04e70593 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:36.997182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:36.997834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:36.998113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:36.998382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:36.999108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.000746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.011604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1147/2331] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:37.015694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.026872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01fc3-a9d7-4175-b88b-6c5dcfe4771f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.031996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.032690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.032966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.033239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.033991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.035719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272418d-9b0e-41c7-8fe4-9fccbe2f3958 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.045970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1185/2332] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:37.050260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.060115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8648f-e609-49c3-beb5-ce158a364b0a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.064913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.065886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.066165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.066438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] PATH_INFO: `/v3/projects/fd8ab1b8c03241fc92511d4d6baa8e5b/users/60be1259db654e2bb24a19d60bf958c3/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.081311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8ab1b8c03241fc92511d4d6baa8e5b, user_id=60be1259db654e2bb24a19d60bf958c3, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.084243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.101361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1148/2333] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:37 2022] PUT /identity/v3/projects/fd8ab1b8c03241fc92511d4d6baa8e5b/users/60be1259db654e2bb24a19d60bf958c3/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:37.111750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.112407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.112682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.139755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] MFA Rules not processed for user `60be1259db654e2bb24a19d60bf958c3`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:37.140985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:37.206194 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1186/2334] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:37.211144 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.273514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094dbd4-8d1c-41e3-aa6a-d17a070f39e7 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.278936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.279594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.279869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.280139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.280859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.282759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.309860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1149/2335] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:20:37 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:37.313960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.324644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb0e5-79fb-43dc-9f28-11670841e504 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.380753 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.381420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.381699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.381969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.382660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.384591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.405116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1187/2336] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:20:37 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:20:37.410654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.421142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc3cc-0fa2-4862-a8f1-0e316a8c4779 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.427035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.427698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.427979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.428271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.429022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.430637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.442466 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1150/2337] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:37.446812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.457802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e5e70-2dc5-4ccb-9539-8919777a0a96 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.463691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.464618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.464891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.465171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] PATH_INFO: `/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2/users/fd24b08a0019480cabc620e0e2e85401/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.480611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d240cd0b4f7485b95548f41d9eec8e2, user_id=fd24b08a0019480cabc620e0e2e85401, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.483255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.498383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1188/2338] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:37 2022] PUT /identity/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2/users/fd24b08a0019480cabc620e0e2e85401/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:37.502507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.512786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4f1c-b568-430d-9603-e01eb4c01a8c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.570474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.571327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.571611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.571887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.572673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.574289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647d08e-4108-4088-ab12-d126d4335815 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.585360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1151/2339] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:37.589516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.600673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d258e2b-406b-40d5-b05d-784fba778a70 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.606124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.607003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.607283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.607561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] PATH_INFO: `/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2/users/fd24b08a0019480cabc620e0e2e85401/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.610277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d240cd0b4f7485b95548f41d9eec8e2, user_id=fd24b08a0019480cabc620e0e2e85401, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.614288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.626716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1189/2340] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:37 2022] PUT /identity/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2/users/fd24b08a0019480cabc620e0e2e85401/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:37.641088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.641424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.641756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.672638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] MFA Rules not processed for user `fd24b08a0019480cabc620e0e2e85401`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:37.673851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:37.756194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1152/2341] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:37.769018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:37.771150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:37.831254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033ab3-5554-4d27-8fe1-f23e106f8e67 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:37.835850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:37.836458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:37.836741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:37.837008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:37.846466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:37.847892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9524b-299d-4764-8cf8-7e3181b9eeed service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:37.859113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1190/2342] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:37.956625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:38.020705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accedc2b-3ae9-4afb-8782-c04e15d8e5b4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:38.025804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:38.026680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:38.026966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:38.027286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] PATH_INFO: `/v3/users/60be1259db654e2bb24a19d60bf958c3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:38.028580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=60be1259db654e2bb24a19d60bf958c3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:38.030547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:38.079875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60be1259db654e2bb24a19d60bf958c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.092937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60be1259db654e2bb24a19d60bf958c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.100017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60be1259db654e2bb24a19d60bf958c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.100725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60be1259db654e2bb24a19d60bf958c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.110914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60be1259db654e2bb24a19d60bf958c3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.116925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1153/2343] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:20:37 2022] DELETE /identity/v3/users/60be1259db654e2bb24a19d60bf958c3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:38.121530 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:38.121817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:38.122092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:38.138564 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:38.139739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:38.201137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1191/2344] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:38.211277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:38.213084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:38.274879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39412218-79fb-4b0f-9258-130ba24c13fc admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:38.279362 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:38.280047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:38.280361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:38.280639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:38.290702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:38.292124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:38.302208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1154/2345] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:20:38.589513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:38.590975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:38.604433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5d27-6792-4ce0-9cd3-bf7714c8f62d None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:38.609819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:38.611224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:38.611567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:38.612691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] PATH_INFO: `/v3/projects/fd8ab1b8c03241fc92511d4d6baa8e5b` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:38.614002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8ab1b8c03241fc92511d4d6baa8e5b)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:38.615918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:38.667176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8ab1b8c03241fc92511d4d6baa8e5b', 'request_id': 'req-50453fe3-9cda-4660-9243-616e40353f4b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.673487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8ab1b8c03241fc92511d4d6baa8e5b', 'request_id': 'req-50453fe3-9cda-4660-9243-616e40353f4b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.681785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8ab1b8c03241fc92511d4d6baa8e5b', 'request_id': 'req-50453fe3-9cda-4660-9243-616e40353f4b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.694696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] The token cache is being invalidate because project fd8ab1b8c03241fc92511d4d6baa8e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:20:38.694978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.695689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.697332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1192/2346] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:20:38 2022] DELETE /identity/v3/projects/fd8ab1b8c03241fc92511d4d6baa8e5b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:38.701518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:38.712627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dccd25-876b-489f-9fc1-ea6afc9dff1d service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:38.778167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:38.779048 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:38.779323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:38.779630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] PATH_INFO: `/v3/users/fd24b08a0019480cabc620e0e2e85401` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:38.780993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd24b08a0019480cabc620e0e2e85401)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:38.782937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:38.841930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd24b08a0019480cabc620e0e2e85401'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.856343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd24b08a0019480cabc620e0e2e85401'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.863408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd24b08a0019480cabc620e0e2e85401'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.864123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd24b08a0019480cabc620e0e2e85401'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.873443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0860e7e6-231e-49d0-b8b3-57d44f0d8449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd24b08a0019480cabc620e0e2e85401'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:38.879858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1155/2347] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:20:38 2022] DELETE /identity/v3/users/fd24b08a0019480cabc620e0e2e85401 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:20:39.172648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.174295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:39.192208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453fe3-9cda-4660-9243-616e40353f4b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.248695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.249570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.249854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.250132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] PATH_INFO: `/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.251323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d240cd0b4f7485b95548f41d9eec8e2)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.253331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.308380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d240cd0b4f7485b95548f41d9eec8e2', 'request_id': 'req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:39.314693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d240cd0b4f7485b95548f41d9eec8e2', 'request_id': 'req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:39.323728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d240cd0b4f7485b95548f41d9eec8e2', 'request_id': 'req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:39.336241 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] The token cache is being invalidate because project 0d240cd0b4f7485b95548f41d9eec8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:20:39.336571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:39.337434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-50c69c9f-be06-4d8b-8755-1bc6461bcd7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:20:39.339330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1193/2348] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:20:39 2022] DELETE /identity/v3/projects/0d240cd0b4f7485b95548f41d9eec8e2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:39.344862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.345379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.345507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.364396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:39.365579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:39.424129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1156/2349] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:39.432133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.432444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.432716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.451556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:39.452786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:39.461803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1194/2350] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:39.466171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.476611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd9b9c-55fe-4f95-bdbd-d7343902fafb None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.481445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.482169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.482457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.482737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.483476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:20:39.483788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.485413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.496784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1157/2351] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:20:39.500980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.511121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db5950-dc86-42aa-9cd9-9306a9e35cb8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.516634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.517280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.517556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.517824 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.518509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.520474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.548775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1195/2352] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:39.553747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.568528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e18fb-14bb-4005-9f08-69a0a016d4af admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.628285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.629237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.629521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.629797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.630467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.632438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.652256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1158/2353] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:39.656363 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.666546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd3ab-2af7-4dd5-8971-49c1e19db378 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.671546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.672235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.672557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.672838 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.673574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.675228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.685835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1196/2354] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:20:39.689876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.700057 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886a702-1672-4498-ad62-08d48bef2383 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.705992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.706653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.706928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.707527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.708506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.710127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1668bf-4b39-4b6d-b912-0098aa124607 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.722590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1159/2355] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:39.727466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.737698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548feab-aac6-4e6d-9738-a13c497439d4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.743003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.743903 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.744188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.744496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] PATH_INFO: `/v3/projects/097299390dbb4a03bdfda9de122fba39/users/8100220e49c24dca8c3d1388e67f989e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.765069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=097299390dbb4a03bdfda9de122fba39, user_id=8100220e49c24dca8c3d1388e67f989e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.767720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:39.783093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1197/2356] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:39 2022] PUT /identity/v3/projects/097299390dbb4a03bdfda9de122fba39/users/8100220e49c24dca8c3d1388e67f989e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:20:39.794043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.794308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.794584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.831942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] MFA Rules not processed for user `8100220e49c24dca8c3d1388e67f989e`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:39.833521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:39.902760 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1160/2357] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:20:39.907423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:39.969533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3c36a-52fe-4746-b5f5-7cd430d557dd admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:39.974981 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:39.975631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:39.975917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:39.976195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:39.976924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:39.978830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.007128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1198/2358] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:20:39 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:40.011187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.022482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de2952-66d8-4048-a9a1-bd81e610e5df None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.095246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.095925 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.096207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.096513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.097274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.099748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.123347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1161/2359] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:20:40 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:20:40.127622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.138335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d5e99-890a-464f-9d9e-dca3d4df43d4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.144160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.144861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.145146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.145421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.146155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.147799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.159226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1199/2360] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:40.163900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.174358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3133b7-a826-4fcd-947d-f0c4842e4d44 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.180651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.181537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.181821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.182098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] PATH_INFO: `/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.198531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f33ee35b9b8478d901b3bf1307a7e84, user_id=6ddbb58d2fa04e74a0c8b9ba9fa963ae, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.201238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.217123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1162/2361] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:40 2022] PUT /identity/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:40.222156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.233000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234024c0-9d39-41aa-b6e9-3ccd67a56164 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.288828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.289495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.289781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.290057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.290789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.292435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f56042-37ac-4b86-add4-ac67050d4514 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.302067 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1200/2362] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:20:40.306564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.317916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429171-f5eb-4ce2-af13-a7d1d77b13ce admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.323516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.324515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.324804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.325077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] PATH_INFO: `/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.327270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f33ee35b9b8478d901b3bf1307a7e84, user_id=6ddbb58d2fa04e74a0c8b9ba9fa963ae, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.329932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.341592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1163/2363] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:20:40 2022] PUT /identity/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:20:40.353804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.354102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.354413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.385016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] MFA Rules not processed for user `6ddbb58d2fa04e74a0c8b9ba9fa963ae`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:20:40.386528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:40.468704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1201/2364] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:20:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:20:40.480076 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:20:40.481868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:20:40.537383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff18105-7a54-4269-a5b6-7b23b9a4a103 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:20:40.541140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:20:40.541784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:20:40.542059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:20:40.542327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:20:40.601081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:20:40.602557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:20:40.611503 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1164/2365] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:20:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:22:22.699834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:22:22.701404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:22:22.782689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37ce4-3c55-4d83-94c5-5e6d6546b411 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:22:22.788134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:22:22.788807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:22:22.789084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:22:22.789354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:22:22.791258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:22:22.804443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:22:22.805992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:22:22.815806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1202/2366] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:22:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:22:36.648012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:22:36.650163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:22:36.756279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ae075-5c05-473c-a5d2-e3f791b90100 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:22:36.762641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:22:36.763293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:22:36.763582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:22:36.763880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:22:36.765811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:22:36.778125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:22:36.779702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:22:36.790019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1165/2367] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:22:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:24:30.443129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:24:30.444855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:24:30.503787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11118ff-cb4d-4bb6-97ab-b9a82134dbd9 service placement] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:24:30.508234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:24:30.508935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:24:30.509198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:24:30.509472 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:24:30.511471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:24:30.521143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:24:30.522571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:24:30.530453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1203/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:24:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:24:35.631519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:24:35.632984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:24:35.646070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1cb04-8857-4524-a684-a90e6caa9bc6 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:24:35.649818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:24:35.650460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:24:35.650774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:24:35.651049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:24:35.652936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:24:35.726034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:24:35.727475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:24:35.737042 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1166/2369] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:24:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:36.011925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:36.021656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149e18d-6862-4d3a-bc33-8e831ca315ff service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:36.026171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:36.026802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:36.027059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:36.027351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:36.029529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:36.129780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:36.133144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:36.153196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1204/2370] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:36.196024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:36.205922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44d7ad-0fdb-4a78-9c62-438b30891881 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:36.211067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:36.211709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:36.211996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:36.212292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:36.222426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:36.223865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:36.233231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1167/2371] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:36.279432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:36.291483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54982-1fd1-4e18-9d15-f12a8001c4cf service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:36.296150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:36.296826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:36.297108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:36.297375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:36.299372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:36.378286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:36.379745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:36.391790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1205/2372] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:36.457667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:36.470013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97348cdf-6805-4587-b1ab-6551f0756fd8 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:36.476677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:36.477601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:36.478004 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:36.478335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:36.481716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:36.557479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:36.558835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:36.577523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1168/2373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:36.740805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:36.756464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace2a24-0a72-44d1-8a7a-08c6517a6dba service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:36.757419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:36.764491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:36.764491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:36.764491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:36.764491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:36.842179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:36.843376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:36.854111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1206/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:37.008065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:37.018399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ac96-c1e3-4062-9fd1-f30ca9220b46 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:37.022931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:37.023607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:37.023892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:37.024173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:37.026225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:37.042761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:37.044352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:37.055683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1169/2375] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:37.270854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:37.272443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:37.283619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd622f9-b14a-4957-be4a-35122ff116ab service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:37.287745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:37.288613 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:37.288960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:37.289313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:37.291197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:37.304806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:37.306319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:37.316419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1207/2376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:26:41.583764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:41.585780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:41.657083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d915bb-6b20-450b-8b6f-5b9ca0ff0396 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:41.665341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:41.667037 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:41.667587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:41.668080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] PATH_INFO: `/v3/users/a71d4761b81447a194e2141442ceef1d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:41.669911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71d4761b81447a194e2141442ceef1d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:41.672415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:41.735523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71d4761b81447a194e2141442ceef1d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:41.751185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71d4761b81447a194e2141442ceef1d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:41.758400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71d4761b81447a194e2141442ceef1d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:41.759246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71d4761b81447a194e2141442ceef1d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:41.768786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71d4761b81447a194e2141442ceef1d'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:41.778142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1170/2377] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:26:41 2022] DELETE /identity/v3/users/a71d4761b81447a194e2141442ceef1d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:26:41.975835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.010546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52964-c37a-4a94-b0cc-e7e77ce9e96d service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.071949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.073246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.073601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.073948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] PATH_INFO: `/v3/projects/d50cb7a7c0d440fca827608bc2a96878` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.075741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50cb7a7c0d440fca827608bc2a96878)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.077786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.139082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.139622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50cb7a7c0d440fca827608bc2a96878', 'request_id': 'req-09b37f6c-da5d-447c-85c6-0d887f315beb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.140622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.147681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50cb7a7c0d440fca827608bc2a96878', 'request_id': 'req-09b37f6c-da5d-447c-85c6-0d887f315beb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.158998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50cb7a7c0d440fca827608bc2a96878', 'request_id': 'req-09b37f6c-da5d-447c-85c6-0d887f315beb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.165442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68a786-bd52-438d-b26e-9a0c277253bb admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.184757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] The token cache is being invalidate because project d50cb7a7c0d440fca827608bc2a96878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:26:42.185092 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.185811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-09b37f6c-da5d-447c-85c6-0d887f315beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.188036 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1208/2378] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:26:41 2022] DELETE /identity/v3/projects/d50cb7a7c0d440fca827608bc2a96878 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:26:42.195612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc16a-9864-415f-9f97-f3940eee055f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.195901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc16a-9864-415f-9f97-f3940eee055f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.196179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc16a-9864-415f-9f97-f3940eee055f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.216999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d3fbc16a-9864-415f-9f97-f3940eee055f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:42.219113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d3fbc16a-9864-415f-9f97-f3940eee055f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.249713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.250763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.251043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.251314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] PATH_INFO: `/v3/users/7facb556fc714c5fac568a21fa5ffcf8` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.252580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7facb556fc714c5fac568a21fa5ffcf8)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.254577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.288414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1209/2379] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:26:42.293843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.295587 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.297128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.319918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7facb556fc714c5fac568a21fa5ffcf8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.328124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:42.329666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.334349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7facb556fc714c5fac568a21fa5ffcf8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.340112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1210/2380] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:26:42.344667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7facb556fc714c5fac568a21fa5ffcf8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.345492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7facb556fc714c5fac568a21fa5ffcf8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.345992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.355885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42339936-eb74-413b-8b9c-5ce149809b92 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.359424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7facb556fc714c5fac568a21fa5ffcf8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.369560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1171/2381] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:26:42 2022] DELETE /identity/v3/users/7facb556fc714c5fac568a21fa5ffcf8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:26:42.369761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.372140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.372460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.372770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.377317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:26:42.377715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.380053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.397165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1211/2382] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:26:42.401139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.402776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.486371 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a753d0-4193-4eb4-a9d2-fbe9caa9a387 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.493156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.494717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.495322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.495770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.497287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.500095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.536577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1172/2383] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:26:42.538099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.552332 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4df70-0b34-40cf-81f4-d15f366f357a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.562397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.565416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.645108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431587cd-7061-48db-afe7-270343cff651 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.650210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.651200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.651559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.651891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] PATH_INFO: `/v3/projects/5d280f7c68b94b91b607a4b1b750cd63` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.653160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d280f7c68b94b91b607a4b1b750cd63)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.655097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.659091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.660045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.660372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.660785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.661579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.663714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.685157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1212/2384] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:26:42.688960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.699776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0174d6-0f9d-46b6-a6e0-63837278e95e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.722639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d280f7c68b94b91b607a4b1b750cd63', 'request_id': 'req-d177651d-875c-449a-9b39-695288b7ee71'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.729022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d280f7c68b94b91b607a4b1b750cd63', 'request_id': 'req-d177651d-875c-449a-9b39-695288b7ee71'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.737564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d280f7c68b94b91b607a4b1b750cd63', 'request_id': 'req-d177651d-875c-449a-9b39-695288b7ee71'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.749210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] The token cache is being invalidate because project 5d280f7c68b94b91b607a4b1b750cd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:26:42.749511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.750186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d177651d-875c-449a-9b39-695288b7ee71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:26:42.752134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1173/2385] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:26:42 2022] DELETE /identity/v3/projects/5d280f7c68b94b91b607a4b1b750cd63 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:26:42.759006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca9dc0-f289-4a44-803d-71ac8b793a72 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.759296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca9dc0-f289-4a44-803d-71ac8b793a72 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.759579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca9dc0-f289-4a44-803d-71ac8b793a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.765588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.766281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.766567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.766968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.767565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.769275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.778158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ebca9dc0-f289-4a44-803d-71ac8b793a72 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:42.779607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ebca9dc0-f289-4a44-803d-71ac8b793a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.779999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1213/2386] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:26:42.784526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.786356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.798220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1174/2387] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:26:42.799646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.799967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.800268 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.802901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa60a66-6b65-453b-8900-1b1189ea8373 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.807920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.808752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.809102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.809448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.810282 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.811959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.820826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:42.822041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.827968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1214/2388] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:26:42.831850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.835561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1175/2389] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:26:42.839789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.841634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119c510-6751-4bd1-9885-d39880d4c273 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.847516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.848574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.848941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.849147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] PATH_INFO: `/v3/projects/d653a013013a4290861ff6d1e6aaefe9/users/e404d6920fca414096387eb64223d62e/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.850603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056de3-404c-45a7-a34b-07b4b8e74ba2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.856850 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.857703 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.857985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.858259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.859047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:26:42.862591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.864275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.865922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d653a013013a4290861ff6d1e6aaefe9, user_id=e404d6920fca414096387eb64223d62e, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.868745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0836-83ea-447e-8467-db1f2ec8f23b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.881665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1176/2390] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:26:42.883561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:42.888674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1215/2391] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:26:42 2022] PUT /identity/v3/projects/d653a013013a4290861ff6d1e6aaefe9/users/e404d6920fca414096387eb64223d62e/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:26:42.896023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3c78d-b94f-4611-9e98-b59c32b243b0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:42.899878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.900164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.900466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.927109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] MFA Rules not processed for user `e404d6920fca414096387eb64223d62e`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:42.928478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:42.965213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:42.965914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:42.966213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:42.966478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:42.967171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:42.969130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:42.997181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1177/2392] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:26:42.998479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1216/2393] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:26:43.003522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:43.009624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:43.011435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:43.073928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf14f80-2334-4baa-a79f-f37726a10e2c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:43.079202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:43.079881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.080404 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.080550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.081140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:43.083279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:43.102670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de096417-1b91-4dc7-9115-4ba7c1ae77b1 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:43.106798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1178/2394] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:26:43 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:26:43.110779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:43.121543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867f287-f201-4af2-8974-4ba8a8a5ab96 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:43.126585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:43.127260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.127530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.127800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.128558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:43.130203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:43.141827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1179/2395] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:26:43.148158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:43.158575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f33999-4691-4779-998c-1c6dc563eec1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:43.164300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:43.165119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.165399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.165668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.166393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:43.168111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:43.176926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:43.178227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.178586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.178963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.179663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1180/2396] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:26:43.183523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:26:43.198738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46064fc-1cf7-44df-af1c-d031924b08c6 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:26:43.205255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:26:43.206133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.206409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.206787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] PATH_INFO: `/v3/projects/c4542d2676e24a4c98191283ad0d5a55/users/5a2bc7ead0714d0cad762186a6421ad3/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.226282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4542d2676e24a4c98191283ad0d5a55, user_id=5a2bc7ead0714d0cad762186a6421ad3, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:43.229696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa17ffc-bded-4e85-8879-c47cb106ecb5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:43.244274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:26:43.254530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1181/2397] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:26:43 2022] PUT /identity/v3/projects/c4542d2676e24a4c98191283ad0d5a55/users/5a2bc7ead0714d0cad762186a6421ad3/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:26:43.254735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2eee75-a39f-442e-9e73-acc0693ee091 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:26:43.255971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.256271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.256570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.261660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1217/2398] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:26:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:26:43.283753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] MFA Rules not processed for user `5a2bc7ead0714d0cad762186a6421ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:43.285039 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:43.353324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1182/2399] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:26:43.706355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:26:43.706642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:26:43.706938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:26:43.723992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] MFA Rules not processed for user `6a41fc2b6e1c4032bfa408eaf36887ee`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:26:43.725383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:26:43.789594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1218/2400] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:26:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:27:24.713329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:24.714833 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:24.828085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47386bb-1f5e-4454-806c-ea778270978f None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:24.833731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:24.834381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:24.834664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:24.834937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:24.837030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:24.849440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:24.851023 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f46282-b257-4936-b203-9f88b6e0bcf0 service placement] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:24.860873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1183/2401] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:27:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:27:43.074031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.162261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e321d7-fc99-4b96-baa7-21f36b975252 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:43.167211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:43.168081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.168387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.168663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] PATH_INFO: `/v3/users/e404d6920fca414096387eb64223d62e` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.169880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e404d6920fca414096387eb64223d62e)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:43.171792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:43.220032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e404d6920fca414096387eb64223d62e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.231702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e404d6920fca414096387eb64223d62e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.232434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e404d6920fca414096387eb64223d62e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.233029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e404d6920fca414096387eb64223d62e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.239659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e404d6920fca414096387eb64223d62e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.253537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1219/2402] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:27:43 2022] DELETE /identity/v3/users/e404d6920fca414096387eb64223d62e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:27:43.258323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.258815 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.258924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.285109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:43.286294 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.346736 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1184/2403] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:43.359796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.361348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.418051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834ec41-e81a-441a-a14d-9ae7389c6645 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:43.422328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:43.423020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.424588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.424588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.428645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.430155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.436232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:43.437737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9dd3e-7434-4fd9-8120-a22f6d1086bb service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:43.442248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ece7-8ca1-4cba-9989-ce3c35419536 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:43.447061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:43.448188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1220/2404] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:27:43.448476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.448605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.448836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] PATH_INFO: `/v3/users/5a2bc7ead0714d0cad762186a6421ad3` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.450105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2bc7ead0714d0cad762186a6421ad3)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:43.452022 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:43.502123 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2bc7ead0714d0cad762186a6421ad3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.514719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2bc7ead0714d0cad762186a6421ad3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.522149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2bc7ead0714d0cad762186a6421ad3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.522906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2bc7ead0714d0cad762186a6421ad3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.530879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2bc7ead0714d0cad762186a6421ad3'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.537236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1185/2405] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:27:43 2022] DELETE /identity/v3/users/5a2bc7ead0714d0cad762186a6421ad3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:27:43.541923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.542268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.542576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.560354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:43.561693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.633085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1221/2406] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:43.642796 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.644270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.706263 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d62d-af6d-46cd-a88c-5dd72edb0731 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:43.710348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:43.711011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.711305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.711602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.722580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:43.723104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.724312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792028a9-e951-450e-8667-170d79d6c5b1 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:43.724637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.735877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1186/2407] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:27:43.736888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504850e-1e2d-4c64-9234-e656e9e067a7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:43.742897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:43.743867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.744238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.744646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] PATH_INFO: `/v3/projects/d653a013013a4290861ff6d1e6aaefe9` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.745957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d653a013013a4290861ff6d1e6aaefe9)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:43.748108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:43.800419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd653a013013a4290861ff6d1e6aaefe9', 'request_id': 'req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.806801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd653a013013a4290861ff6d1e6aaefe9', 'request_id': 'req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.814768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd653a013013a4290861ff6d1e6aaefe9', 'request_id': 'req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.825104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] The token cache is being invalidate because project d653a013013a4290861ff6d1e6aaefe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:27:43.825396 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.826048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-b6e93ca4-c82d-4134-9139-a732bd9f1dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:43.827654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1222/2408] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:27:43 2022] DELETE /identity/v3/projects/d653a013013a4290861ff6d1e6aaefe9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:27:43.832707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.832996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.833271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.882006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:43.883118 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.941065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1187/2409] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:27:43.949688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:43.950198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:43.950611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:43.970243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:43.971596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:43.982440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1223/2410] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:27:43.987014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:43.997081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9343ae-9c1e-4886-9544-f3b18cb5ca8a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.002397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.003015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.003360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.003708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.004579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:27:44.004884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.006534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.011068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.012550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.021178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1188/2411] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:27:44.025095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.035717 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3342a-1b1f-44f9-bbb8-33cfe672ac59 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.040667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.041306 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.041562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.041841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.042537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.043647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923f155-b83d-4f18-9f05-5424ca360477 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.044520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.048804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.049207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.049493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.049756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] PATH_INFO: `/v3/projects/c4542d2676e24a4c98191283ad0d5a55` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.051062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4542d2676e24a4c98191283ad0d5a55)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.052988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.084913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1189/2412] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.089017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.102609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4542d2676e24a4c98191283ad0d5a55', 'request_id': 'req-146b6ca4-001d-429e-b9ef-d93c3d8a87af'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:44.109044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4542d2676e24a4c98191283ad0d5a55', 'request_id': 'req-146b6ca4-001d-429e-b9ef-d93c3d8a87af'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:44.117858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4542d2676e24a4c98191283ad0d5a55', 'request_id': 'req-146b6ca4-001d-429e-b9ef-d93c3d8a87af'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:44.128532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] The token cache is being invalidate because project c4542d2676e24a4c98191283ad0d5a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:27:44.128750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:44.129418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-146b6ca4-001d-429e-b9ef-d93c3d8a87af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:27:44.131033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1224/2413] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:27:44 2022] DELETE /identity/v3/projects/c4542d2676e24a4c98191283ad0d5a55 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:27:44.135820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dee7fa-5410-457a-86fc-ee895e77fc2a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.136109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dee7fa-5410-457a-86fc-ee895e77fc2a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.136438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dee7fa-5410-457a-86fc-ee895e77fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.151987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710817ad-bdb7-4c43-a3f1-8d6c26e45582 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.154897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-36dee7fa-5410-457a-86fc-ee895e77fc2a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:44.156078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-36dee7fa-5410-457a-86fc-ee895e77fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.206187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.206852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.207127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.207397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.208077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.210115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.214935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1225/2414] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:44.222962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.223256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.223534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.230665 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1190/2415] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.234419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.236115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.242875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:44.244162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.248803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50171a-66b7-4a36-bfee-f23b4c5e348b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.252898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1226/2416] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:44.253679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.254354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.254638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.254915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.255792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.257074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.257739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.266122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a7fe-6fff-4505-90ee-68a0e37260c3 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.268758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1191/2417] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:27:44.271457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.272192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.272518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.272915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.273090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.274206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:27:44.274738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.277359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.283011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771a89-6c29-47a6-901e-cf7d8a65f49b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.288130 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.288821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.289103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.289377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.290109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.291705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.293881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1227/2418] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:27:44.299479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.307289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1192/2419] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:27:44.311535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e78042-a48e-4080-b29e-58f520612d41 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.312198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.316854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.317516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.317803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.318081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.318779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.320757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.321978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09122ec9-a3a0-4606-8bc5-1bac2748fc1e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.326920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.327787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.328067 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.328375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] PATH_INFO: `/v3/projects/6259e1e9b8cf4123a213c7269ba80154/users/59f39e7407514661a5542e9ef8cde7ac/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.342791 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6259e1e9b8cf4123a213c7269ba80154, user_id=59f39e7407514661a5542e9ef8cde7ac, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.345564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0f53-47e3-4589-b756-865dd86b2012 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.348204 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1228/2420] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.352134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.361733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1193/2421] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:44 2022] PUT /identity/v3/projects/6259e1e9b8cf4123a213c7269ba80154/users/59f39e7407514661a5542e9ef8cde7ac/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:27:44.373308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.373569 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.373878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.400514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] MFA Rules not processed for user `59f39e7407514661a5542e9ef8cde7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:44.401689 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.413915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485926f4-50e3-477f-8812-919d74a12857 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.464472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1194/2422] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:44.467722 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.468512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.468787 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.468944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.469041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.470869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.472842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.479231 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ce6ac-34ab-499f-8a55-9fb5df7a1062 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.483909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.484602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.484903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.485205 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.485944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.487822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.493889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1229/2423] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.497998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.508652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030be074-c571-4f46-bf92-a90ce37cea31 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.513786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.514445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.514730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.515006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.515772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.517447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.520722 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1195/2424] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.521618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.531239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1230/2425] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:27:44.532267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.541833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c61e6-cd69-405e-a128-2888f4586299 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.588033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78331-64d6-4608-9cc1-0aa24b971200 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.593161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.593986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.594274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.594889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.595594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.597624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.602815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.603490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.603828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.604107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.604870 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.606485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.617130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1231/2426] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:27:44.621291 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1196/2427] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.621652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.622316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.631624 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec60f6c-9a36-41c9-95d1-0f37c9a37144 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.632132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8f54-4cf4-48e0-bc46-df65e71e1c82 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.636357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.636769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.637227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.637486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.637651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.637772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.637926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] PATH_INFO: `/v3/projects/707b57693203419783ab4eb8d3d7affd/users/61da3a300d2046f4873f4e82734b3e1f/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.638049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.638758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.640378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.650625 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1197/2428] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:27:44.652643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707b57693203419783ab4eb8d3d7affd, user_id=61da3a300d2046f4873f4e82734b3e1f, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.654451 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.655445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6a27e-593f-4a4c-957c-93c1f8089562 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.664442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6d7d-cdc4-435a-b92b-6e557731e97f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.669209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1232/2429] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:44 2022] PUT /identity/v3/projects/707b57693203419783ab4eb8d3d7affd/users/61da3a300d2046f4873f4e82734b3e1f/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:27:44.679916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.680210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.680524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.707203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] MFA Rules not processed for user `61da3a300d2046f4873f4e82734b3e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:44.708484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.720063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.720995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.721282 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.721559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] PATH_INFO: `/v3/projects/aa6300bbaa754a6599079cff2567b7c8/users/af916674c9e14564b4cbd2ea5a11f04f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.735904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6300bbaa754a6599079cff2567b7c8, user_id=af916674c9e14564b4cbd2ea5a11f04f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.738712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.752719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1198/2430] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:44 2022] PUT /identity/v3/projects/aa6300bbaa754a6599079cff2567b7c8/users/af916674c9e14564b4cbd2ea5a11f04f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:27:44.756566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.773215 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1233/2431] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:44.777655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.787545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b9e80-43c7-4f0e-8e01-4f98666520a5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.816149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d446-f552-4e16-ad7d-c71e28134363 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.820856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.821524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.821807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.822081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.823181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.824816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.834663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1199/2432] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:27:44.838872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.843513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.844205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.844685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.844968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.845664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.847630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f198c12-3784-4ca3-a661-dd566f5ba5bf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.848179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0912c8-f632-4266-9a81-1c47eb2ad295 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.852798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.853645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.853922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.854195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] PATH_INFO: `/v3/projects/aa6300bbaa754a6599079cff2567b7c8/users/af916674c9e14564b4cbd2ea5a11f04f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.856151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6300bbaa754a6599079cff2567b7c8, user_id=af916674c9e14564b4cbd2ea5a11f04f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.858753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:44.870089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1200/2433] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:44 2022] PUT /identity/v3/projects/aa6300bbaa754a6599079cff2567b7c8/users/af916674c9e14564b4cbd2ea5a11f04f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:27:44.874541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1234/2434] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:44.879810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:44.880276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.880737 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.881141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.907339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] MFA Rules not processed for user `af916674c9e14564b4cbd2ea5a11f04f`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:44.908627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:44.939930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf442bc-3c4d-4454-8d52-d2cc1b0a4fde admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:44.974496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1235/2435] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:27:44.994757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:44.995431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:44.995710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:44.995979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:44.996692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:44.998600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:45.020836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1201/2436] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:27:44 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:27:45.026198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:45.036985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72a8ff-1797-4873-abc9-d74f95b79cb8 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:45.043237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:45.043909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:45.044210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:45.044486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:45.045205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:45.046816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:45.057448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1236/2437] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:27:45.061810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:45.072589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006e86d-eada-42d7-bd9e-4c985d1a0292 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:45.078276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:45.079243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:45.079534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:45.079808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] PATH_INFO: `/v3/projects/68904d81896748c58b451f94b297891f/users/946e27a0414548158d5b8f41d3cf70a3/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:45.095396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68904d81896748c58b451f94b297891f, user_id=946e27a0414548158d5b8f41d3cf70a3, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:45.098109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:45.113074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1202/2438] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:45 2022] PUT /identity/v3/projects/68904d81896748c58b451f94b297891f/users/946e27a0414548158d5b8f41d3cf70a3/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:27:45.119060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:45.131801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a860bb5-baf3-4088-a494-e2a8ec9d2b69 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:45.188476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:45.189077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:45.189353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:45.189626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:45.190368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:45.191972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4f89a-375c-49ae-a094-0b1d34fa827d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:45.202349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1237/2439] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:27:45.207202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:27:45.222365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c371-eb3f-4cb7-ac66-2e9deb2a1a6a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:27:45.227644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:27:45.228558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:45.228838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:45.229126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] PATH_INFO: `/v3/projects/68904d81896748c58b451f94b297891f/users/946e27a0414548158d5b8f41d3cf70a3/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:45.231252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68904d81896748c58b451f94b297891f, user_id=946e27a0414548158d5b8f41d3cf70a3, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:27:45.233948 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:27:45.244927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1203/2440] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:27:45 2022] PUT /identity/v3/projects/68904d81896748c58b451f94b297891f/users/946e27a0414548158d5b8f41d3cf70a3/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:27:45.257873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:27:45.258161 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:27:45.258435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:27:45.286262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] MFA Rules not processed for user `946e27a0414548158d5b8f41d3cf70a3`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:27:45.287464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:27:45.355745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1238/2441] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:27:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:31.888781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:31.891817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:31.969952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87569475-6380-4e92-80b8-7cd6a63ac02b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:31.973798 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:31.974480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:31.974757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:31.975027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.038075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:32.039768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:32.051920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1204/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:32.084871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:32.086826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:32.120599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc0097-77e6-4658-addf-29f151e2bd29 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:32.125004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:32.125665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:32.125947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:32.126479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.137355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:32.139097 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:32.158573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1239/2443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:32.256138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:32.257882 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:32.294070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:32.304079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5350173-6d04-4ec3-9b85-7573d9c20433 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:32.310195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:32.311001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:32.311380 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:32.311775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.353804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd42c53-3c83-4abc-b934-79342bc17d79 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:32.359399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:32.360046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:32.360352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:32.360630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.362476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:32.376516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] Could not find project: c4542d2676e24a4c98191283ad0d5a55.: keystone.exception.ProjectNotFound: Could not find project: c4542d2676e24a4c98191283ad0d5a55. May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 12 01:28:32.377477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 01:28:32.378672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self._validate_token_resources() May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 01:28:32.380907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.get_or_create( May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application with Lock( May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._enter() May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.creator() May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application created_value = creator( May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 01:28:32.382967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 01:28:32.385813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c4542d2676e24a4c98191283ad0d5a55. May 12 01:28:32.385813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: ERROR keystone.server.flask.application May 12 01:28:32.386002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:32.386109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1205/2444] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:32 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 123 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 12 01:28:32.386109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:32.394334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d803-881a-4c0d-a9f2-abfbcb180943 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:32.398832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:32.400013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:32.400524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:32.400973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.412625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1240/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:32.487267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:32.488526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:32.498733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1206/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:32.786051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:32.787457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:32.854969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a703-abee-4864-9cfe-8488deaf3c62 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:32.858667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:32.859311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:32.859588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:32.859860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:32.861639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:32.871200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:32.872665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c222e38-2979-4b7b-bd61-09502abbc845 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:32.880720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1241/2447] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:33.320670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:33.322749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.391561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e834f-6f2d-4001-afd8-dee9bbb8ab5c service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:33.396540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:33.397424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:33.397701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:33.397970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] PATH_INFO: `/v3/users/8100220e49c24dca8c3d1388e67f989e` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:33.399177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8100220e49c24dca8c3d1388e67f989e)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:33.401171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:33.451725 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8100220e49c24dca8c3d1388e67f989e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:33.464867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8100220e49c24dca8c3d1388e67f989e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:33.473445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8100220e49c24dca8c3d1388e67f989e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:33.474694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8100220e49c24dca8c3d1388e67f989e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:33.482742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8100220e49c24dca8c3d1388e67f989e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:33.489377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1207/2448] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:33 2022] DELETE /identity/v3/users/8100220e49c24dca8c3d1388e67f989e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:33.493812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:33.494315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:33.494572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:33.513105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:33.514382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.581920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1242/2449] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:33.590449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:33.592307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.651851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:33.653404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.667972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc236-4a61-4bff-b295-4bc71dd190c4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:33.672975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:33.673786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:33.674069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:33.674449 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:33.700388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:33.702544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:33.721305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1208/2450] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:33.726609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:33.730332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.751279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5a39-0d67-4c36-aaff-311a85c590c5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:33.757156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:33.757991 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:33.758447 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:33.758904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:33.761167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.778898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:33.781275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:33.793187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1243/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:33.863032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47993bb-4413-4d57-af45-65d9ebc3255c service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:33.878139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:33.878139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:33.878139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:33.878139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:33.879190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:33.911747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:33.913716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:33.926127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1209/2452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:34.046478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:34.057533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5cd6d-04e8-4686-b643-13c967ac98d8 service placement] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:34.069143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:34.070090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:34.070547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:34.071276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:34.075704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:34.098174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:34.100442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:34.113745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1244/2453] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:34.589757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:34.592276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:34.612795 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142f71-b124-415a-a355-c1ee6cda7f10 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:34.619652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:34.620821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:34.621745 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:34.622524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] PATH_INFO: `/v3/projects/097299390dbb4a03bdfda9de122fba39` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:34.624319 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097299390dbb4a03bdfda9de122fba39)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:34.627139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:34.737288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097299390dbb4a03bdfda9de122fba39', 'request_id': 'req-7da71543-e2d3-4dbd-b749-a2b83bcc5421'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.745141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097299390dbb4a03bdfda9de122fba39', 'request_id': 'req-7da71543-e2d3-4dbd-b749-a2b83bcc5421'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.757310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097299390dbb4a03bdfda9de122fba39', 'request_id': 'req-7da71543-e2d3-4dbd-b749-a2b83bcc5421'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.772472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] The token cache is being invalidate because project 097299390dbb4a03bdfda9de122fba39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:34.773089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.773905 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.775898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1210/2454] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:34 2022] DELETE /identity/v3/projects/097299390dbb4a03bdfda9de122fba39 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:34.780781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:34.795100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30768665-0486-4c01-b87e-ca180b3fdaa7 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:34.857724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:34.858801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:34.859046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:34.859344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] PATH_INFO: `/v3/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:34.860633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddbb58d2fa04e74a0c8b9ba9fa963ae)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:34.862664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:34.915429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddbb58d2fa04e74a0c8b9ba9fa963ae'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.926580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddbb58d2fa04e74a0c8b9ba9fa963ae'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.927368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddbb58d2fa04e74a0c8b9ba9fa963ae'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.928001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddbb58d2fa04e74a0c8b9ba9fa963ae'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.934905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddbb58d2fa04e74a0c8b9ba9fa963ae'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:34.949238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1245/2455] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:34 2022] DELETE /identity/v3/users/6ddbb58d2fa04e74a0c8b9ba9fa963ae => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:35.047687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.050976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.079255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da71543-e2d3-4dbd-b749-a2b83bcc5421 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.190832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.193262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.193520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.193844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] PATH_INFO: `/v3/users/2b6a0a26494e4e619d504f10ef20fb3b` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.196170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6a0a26494e4e619d504f10ef20fb3b)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.198255 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.284723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6a0a26494e4e619d504f10ef20fb3b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.298102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6a0a26494e4e619d504f10ef20fb3b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.309506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6a0a26494e4e619d504f10ef20fb3b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.311663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6a0a26494e4e619d504f10ef20fb3b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.327199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6a0a26494e4e619d504f10ef20fb3b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.337946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1211/2456] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:35 2022] DELETE /identity/v3/users/2b6a0a26494e4e619d504f10ef20fb3b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:35.421667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.423113 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.439058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f181f-e4bc-4044-b13e-29e35daa072d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.528480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.529422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.529703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.530174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] PATH_INFO: `/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.531595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f33ee35b9b8478d901b3bf1307a7e84)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.533813 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.547105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.548792 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.562757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b93455-68c4-4c5a-94cf-cefc885a193a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.569535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.570461 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.570775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.571053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] PATH_INFO: `/v3/projects/5eab5b22675c4c7bacddb9659bed29c2` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.572967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eab5b22675c4c7bacddb9659bed29c2)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.578253 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.590442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f33ee35b9b8478d901b3bf1307a7e84', 'request_id': 'req-9db483e6-f4ca-44f3-86bc-37475c35ab3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.597702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f33ee35b9b8478d901b3bf1307a7e84', 'request_id': 'req-9db483e6-f4ca-44f3-86bc-37475c35ab3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.608893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f33ee35b9b8478d901b3bf1307a7e84', 'request_id': 'req-9db483e6-f4ca-44f3-86bc-37475c35ab3f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.626867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] The token cache is being invalidate because project 4f33ee35b9b8478d901b3bf1307a7e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:35.627301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.628395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-9db483e6-f4ca-44f3-86bc-37475c35ab3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.632024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1246/2457] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:35 2022] DELETE /identity/v3/projects/4f33ee35b9b8478d901b3bf1307a7e84 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:35.637784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6ece-b90a-49d9-9545-4a573b43318a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.638837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6ece-b90a-49d9-9545-4a573b43318a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.639191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6ece-b90a-49d9-9545-4a573b43318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.652600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eab5b22675c4c7bacddb9659bed29c2', 'request_id': 'req-6b241a9d-5ffc-4277-babd-43cd5f918d07'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.659601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8cdd6ece-b90a-49d9-9545-4a573b43318a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:35.661126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8cdd6ece-b90a-49d9-9545-4a573b43318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.661567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eab5b22675c4c7bacddb9659bed29c2', 'request_id': 'req-6b241a9d-5ffc-4277-babd-43cd5f918d07'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.671169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eab5b22675c4c7bacddb9659bed29c2', 'request_id': 'req-6b241a9d-5ffc-4277-babd-43cd5f918d07'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.682595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] The token cache is being invalidate because project 5eab5b22675c4c7bacddb9659bed29c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:35.682907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.683542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6b241a9d-5ffc-4277-babd-43cd5f918d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:35.685418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1212/2458] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:35 2022] DELETE /identity/v3/projects/5eab5b22675c4c7bacddb9659bed29c2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:35.692340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916788ac-5899-44e9-98e0-5f1a6314c20f None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.692626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916788ac-5899-44e9-98e0-5f1a6314c20f None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.692902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916788ac-5899-44e9-98e0-5f1a6314c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.712096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-916788ac-5899-44e9-98e0-5f1a6314c20f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:35.713468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-916788ac-5899-44e9-98e0-5f1a6314c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.726791 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1247/2459] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:35.735688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.736052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.736430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.761451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:35.762879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.775629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1248/2460] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:35.780871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.790662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c19076-6296-47d0-abd5-06b955aba896 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.802368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1213/2461] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:35.802572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.803306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.803600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.803863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.804664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:35.805134 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.807326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.809680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.810698 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.811802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.822648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1249/2462] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:28:35.827010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.839077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786f35-50a0-433c-b9a2-d8e9273a279f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.844182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.845232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.845799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.846144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.846893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.849024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.849727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:35.850934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:35.862769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1214/2463] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:35.869638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.882858 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1250/2464] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:35.884018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd0dda-e356-459d-9be1-b615906dc58d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.887665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:35.966971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.968044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.968592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.969017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.970355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:35.970899 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.972361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d80c-8c91-4fb4-9912-e37c505aaa3b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:35.972794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.980311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:35.981049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:35.981329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:35.981599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:35.982655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:35.985664 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:35.993542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1215/2465] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:28:35.998168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.011879 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ae2f0-35ce-46ae-9708-7d995cb214c3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.023599 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1251/2466] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:36.025208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.026755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.027146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.027377 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.027998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.029102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.031808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.048614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407534f-6836-472d-b9ee-1734ab56c4e2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.055705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.056406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.056863 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.057293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.058289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.061793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.079258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1216/2467] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:35 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:36.082783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.083024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1252/2468] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:28:36.085172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.101061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b740a4-aa42-460d-b0c6-a4038c025b9c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.182660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.183414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.183835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.184115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.184836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.186881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.208441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8a287-c80d-4b8f-b649-2df3e3115483 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.215070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1253/2469] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:36 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:36.218047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.218209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.222375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.222375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.222375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.222375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.223865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.232473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca5aa-4fb4-49ac-a8d0-f985f9e52112 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.240064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1217/2470] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:36.241364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.245125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.245707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.252480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.252786 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.253579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.255355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.255838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d236853-4984-4fdb-a6b9-68cf3cc571dd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.271781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1254/2471] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:36.275160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.275366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.276497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.277595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.277973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] PATH_INFO: `/v3/projects/3d2f1b69a8e94695a6db56db0e935384/users/9442d41faa954f2dacd31c26051ffd06/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.297004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0df4-f2da-4a8a-a543-f732aa04cb6f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.309020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2f1b69a8e94695a6db56db0e935384, user_id=9442d41faa954f2dacd31c26051ffd06, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.311125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.311823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.312115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.312559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.313304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.313612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.318767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257e441-1196-44ac-a301-8270605180a9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.333358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1218/2472] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:36 2022] PUT /identity/v3/projects/3d2f1b69a8e94695a6db56db0e935384/users/9442d41faa954f2dacd31c26051ffd06/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:36.341456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1255/2473] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:36.343165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.397340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.397614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.397876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.434473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] MFA Rules not processed for user `9442d41faa954f2dacd31c26051ffd06`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:36.435861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.444036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68091a-1410-49aa-b9d3-182af0f20328 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.449633 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.453480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.453480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.453480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] PATH_INFO: `/v3/projects/d5bc257e76d54d0abc67c39792122b0d/users/4b84b7192b13421286d86ba3a43d0475/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.486013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bc257e76d54d0abc67c39792122b0d, user_id=4b84b7192b13421286d86ba3a43d0475, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.489636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388d87-a319-4e74-8f53-7b7e9e9a580a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.509758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1219/2474] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:36 2022] PUT /identity/v3/projects/d5bc257e76d54d0abc67c39792122b0d/users/4b84b7192b13421286d86ba3a43d0475/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:36.515015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.515344 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.515627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.516709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1256/2475] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:36.531637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.534315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.540873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:36.542272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.604854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1220/2476] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:36.608379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.608379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.618137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d5bbd-864e-47de-8e8e-93918061544a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.622704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.623573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.623852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.624124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] PATH_INFO: `/v3/users/61da3a300d2046f4873f4e82734b3e1f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.625347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=61da3a300d2046f4873f4e82734b3e1f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.627254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.632605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b56831-e2c0-4345-9910-9432433e6f75 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.677750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61da3a300d2046f4873f4e82734b3e1f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:36.688191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.688874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.689154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.689430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.689824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61da3a300d2046f4873f4e82734b3e1f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:36.696863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61da3a300d2046f4873f4e82734b3e1f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:36.697642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61da3a300d2046f4873f4e82734b3e1f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:36.707664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61da3a300d2046f4873f4e82734b3e1f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:36.714169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1221/2477] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:36 2022] DELETE /identity/v3/users/61da3a300d2046f4873f4e82734b3e1f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:36.714868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.717021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.750146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.751614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0170b6-d053-4244-abcb-591a973dff63 service glance] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.760402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1257/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:36.761665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.761965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.762213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.772869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dffc85-4849-4288-9d9c-eb615bb6a92c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.776684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.777552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.777834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.778106 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.780714 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:36.781954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.783410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.850781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.852313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.852441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1258/2479] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:36.859725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:36.861176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:36.863327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1222/2480] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:36.874166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d328a-75ff-4bb9-a608-eabfffc4c4b5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:36.878591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:36.879317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:36.879687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:36.880182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:36.890768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:36.892214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:36.909967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1259/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:37.271763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:37.273349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:37.293581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc01473-6fbe-4cf5-afa9-2ae7533f8faf service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:37.315368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:37.315884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:37.316238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:37.316611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] PATH_INFO: `/v3/projects/707b57693203419783ab4eb8d3d7affd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:37.317972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=707b57693203419783ab4eb8d3d7affd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:37.319988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:37.376775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707b57693203419783ab4eb8d3d7affd', 'request_id': 'req-f1db4c9a-f74b-4e99-a74a-5d0792770b97'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.383389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707b57693203419783ab4eb8d3d7affd', 'request_id': 'req-f1db4c9a-f74b-4e99-a74a-5d0792770b97'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.391072 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707b57693203419783ab4eb8d3d7affd', 'request_id': 'req-f1db4c9a-f74b-4e99-a74a-5d0792770b97'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.401727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] The token cache is being invalidate because project 707b57693203419783ab4eb8d3d7affd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:37.402015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.402642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.404233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1223/2482] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:37 2022] DELETE /identity/v3/projects/707b57693203419783ab4eb8d3d7affd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:37.408229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:37.417674 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392e41a-00a8-4e54-a778-3b23ebd59c25 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:37.473256 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:37.474327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:37.474629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:37.474899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] PATH_INFO: `/v3/users/946e27a0414548158d5b8f41d3cf70a3` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:37.476830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] RBAC: Authorizing `identity:delete_user(user_id=946e27a0414548158d5b8f41d3cf70a3)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:37.478892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:37.525524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946e27a0414548158d5b8f41d3cf70a3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.537727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946e27a0414548158d5b8f41d3cf70a3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.538444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946e27a0414548158d5b8f41d3cf70a3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.539115 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946e27a0414548158d5b8f41d3cf70a3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.546501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-24b0052d-f511-407c-9783-9003de6bf13c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946e27a0414548158d5b8f41d3cf70a3'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:37.559232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1260/2483] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:37 2022] DELETE /identity/v3/users/946e27a0414548158d5b8f41d3cf70a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:37.855241 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:37.856834 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:37.871505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4c9a-f74b-4e99-a74a-5d0792770b97 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:37.936160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:37.937075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:37.937366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:37.937643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] PATH_INFO: `/v3/projects/68904d81896748c58b451f94b297891f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:37.938860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68904d81896748c58b451f94b297891f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:37.940888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:37.996707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68904d81896748c58b451f94b297891f', 'request_id': 'req-255fc111-d7ec-4934-ae8b-7ac8106cd072'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:38.003383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68904d81896748c58b451f94b297891f', 'request_id': 'req-255fc111-d7ec-4934-ae8b-7ac8106cd072'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:38.013704 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68904d81896748c58b451f94b297891f', 'request_id': 'req-255fc111-d7ec-4934-ae8b-7ac8106cd072'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:38.026521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] The token cache is being invalidate because project 68904d81896748c58b451f94b297891f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:38.026881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:38.027497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-255fc111-d7ec-4934-ae8b-7ac8106cd072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:38.032794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1224/2484] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:37 2022] DELETE /identity/v3/projects/68904d81896748c58b451f94b297891f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:38.034454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.034954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.035087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.054438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:38.055652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:38.128370 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1261/2485] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:38.135937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.136225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.136534 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.158983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:38.161006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:38.170705 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1225/2486] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:38.175156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.187318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5dae9-7684-4bca-9e08-f36cf4896293 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.193670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.194006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.194423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.194979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.196287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:38.197195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.199201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.220651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1262/2487] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:28:38.221914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.231921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aea5d2-5490-48d3-b7ed-60524f2a2cc9 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.237998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.239680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.239680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.239680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.240353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.242340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.272300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1226/2488] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:38.275103 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.285552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b08c-fb0a-4200-a0e9-3afbf268bd54 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.361888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.362050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.362285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.362566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.363279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.365496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.400628 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1263/2489] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:38.401992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.420393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068fc9-3976-491d-b3d9-eb91e63aba0e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.428714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.429478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.429832 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.430174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.432889 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.434839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.448900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1227/2490] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:38.455799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.469220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d6e-cd18-4bee-b8ab-daf1ca6db87e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.474565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.475335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.475670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.475947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.477030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.478645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2db9b-bb43-4f42-803b-1ba1adea1407 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.492685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1264/2491] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:38.496121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.508179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29393-a90e-4b45-be61-42161146a5dd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.515289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.516467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.516923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.517295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] PATH_INFO: `/v3/projects/a486281494d24f21b102bf6a19682050/users/c81c6da7029943369932cb164b72b9c8/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.536806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a486281494d24f21b102bf6a19682050, user_id=c81c6da7029943369932cb164b72b9c8, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.540837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.558607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1228/2492] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:38 2022] PUT /identity/v3/projects/a486281494d24f21b102bf6a19682050/users/c81c6da7029943369932cb164b72b9c8/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:38.570166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.570440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.570738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.615024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] MFA Rules not processed for user `c81c6da7029943369932cb164b72b9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:38.615760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:38.708432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1265/2493] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:38.712863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.794527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a91c9b-96be-4a56-ba83-a66956f714a8 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.800200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.800908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.801219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.801494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.802212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.804168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.834102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1229/2494] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:38.835393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.846082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b11dc-e58b-4a6c-a24d-68f35f7d93a5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.902683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.903355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.903662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.904073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.904678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.906806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.927844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1266/2495] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:38 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:38.932027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.943181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72075e1e-2dd2-4f6b-8d96-b6bb1dae2a40 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.948652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.949335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.949621 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.949897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:38.950700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:38.952398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:38.964724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1230/2496] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:38.969046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:38.979337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef1f9a-f96b-49b4-a7c8-f379cc242de8 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:38.984423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:38.985329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:38.985615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:38.985893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] PATH_INFO: `/v3/projects/a39f49ba060b4bea879adba92a34809c/users/66f53064d88e4a5cb5a9106426fe9ba7/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.001552 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39f49ba060b4bea879adba92a34809c, user_id=66f53064d88e4a5cb5a9106426fe9ba7, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:39.004235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:39.018170 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1267/2497] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:38 2022] PUT /identity/v3/projects/a39f49ba060b4bea879adba92a34809c/users/66f53064d88e4a5cb5a9106426fe9ba7/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:39.022215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:39.032292 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5db91-7be5-42b6-a872-ff286efc55d8 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:39.090817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:39.091927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.092518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.092518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.093120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:39.094742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7001965-1626-4f8b-9553-2ce380c6a5d7 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:39.105842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1231/2498] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:39.110755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:39.122009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461ab4-4650-4813-a163-ab7ce1d0d56f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:39.128461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:39.129375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.129659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.129968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] PATH_INFO: `/v3/projects/a39f49ba060b4bea879adba92a34809c/users/66f53064d88e4a5cb5a9106426fe9ba7/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.131957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39f49ba060b4bea879adba92a34809c, user_id=66f53064d88e4a5cb5a9106426fe9ba7, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:39.134647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:39.150000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1268/2499] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:39 2022] PUT /identity/v3/projects/a39f49ba060b4bea879adba92a34809c/users/66f53064d88e4a5cb5a9106426fe9ba7/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:39.162430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.162723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.163003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.192676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] MFA Rules not processed for user `66f53064d88e4a5cb5a9106426fe9ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:39.193961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:39.269218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1232/2500] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:39.283015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:39.285659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:39.380205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb663c-7278-4033-a076-74abc42bed27 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:39.384364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:39.385032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.385308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.385621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.397818 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:39.400208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451303e-ec52-4e72-b067-b13cd982525f service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:39.412793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1269/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:39.601642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:39.673453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306cb40-278f-421d-959e-ab137b77e942 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:39.678256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:39.679167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.679448 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.679726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] PATH_INFO: `/v3/users/c81c6da7029943369932cb164b72b9c8` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.680958 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81c6da7029943369932cb164b72b9c8)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:39.682894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:39.737653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81c6da7029943369932cb164b72b9c8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:39.751196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81c6da7029943369932cb164b72b9c8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:39.774984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81c6da7029943369932cb164b72b9c8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:39.776329 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81c6da7029943369932cb164b72b9c8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:39.799407 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81c6da7029943369932cb164b72b9c8'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:39.807835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1233/2502] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:39 2022] DELETE /identity/v3/users/c81c6da7029943369932cb164b72b9c8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:39.812792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:39.813494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:39.813994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:39.836043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:39.837658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:39.909600 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1270/2503] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:39.920907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:39.922475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:39.992603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a522408-809f-45b7-aa34-81b794183d60 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:39.997863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:39.999516 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:40.000053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:40.000699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:40.011786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:40.013921 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:40.032007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1234/2504] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:40.460073 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:40.461763 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:40.494985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d9770-b9d1-4865-a55e-02be59dc1688 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:40.503642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:40.504992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:40.505436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:40.505875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] PATH_INFO: `/v3/projects/a486281494d24f21b102bf6a19682050` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:40.507317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a486281494d24f21b102bf6a19682050)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:40.509794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:40.644069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a486281494d24f21b102bf6a19682050', 'request_id': 'req-a4edf178-32ae-4a75-b277-d34b9da1376a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:40.658023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a486281494d24f21b102bf6a19682050', 'request_id': 'req-a4edf178-32ae-4a75-b277-d34b9da1376a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:40.678762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a486281494d24f21b102bf6a19682050', 'request_id': 'req-a4edf178-32ae-4a75-b277-d34b9da1376a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:40.713270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] The token cache is being invalidate because project a486281494d24f21b102bf6a19682050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:40.714108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:40.715728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:40.721306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1271/2505] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:40 2022] DELETE /identity/v3/projects/a486281494d24f21b102bf6a19682050 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:40.725408 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:40.736183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43970-d3c6-4bec-abd0-0d27dd949efa service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:40.903325 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:40.904652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:40.905104 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:40.905670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] PATH_INFO: `/v3/users/66f53064d88e4a5cb5a9106426fe9ba7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:40.907893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f53064d88e4a5cb5a9106426fe9ba7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:40.910060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:40.985178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f53064d88e4a5cb5a9106426fe9ba7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.001754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f53064d88e4a5cb5a9106426fe9ba7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.011779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f53064d88e4a5cb5a9106426fe9ba7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.017603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f53064d88e4a5cb5a9106426fe9ba7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.032035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b946b6dd-7d3c-4eb4-8bf9-252754f4b702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f53064d88e4a5cb5a9106426fe9ba7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.039145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1235/2506] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:40 2022] DELETE /identity/v3/users/66f53064d88e4a5cb5a9106426fe9ba7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:41.389807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:41.392074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:41.426507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edf178-32ae-4a75-b277-d34b9da1376a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:41.532643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:41.534093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:41.534376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:41.535078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] PATH_INFO: `/v3/projects/a39f49ba060b4bea879adba92a34809c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:41.536385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39f49ba060b4bea879adba92a34809c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:41.538823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:41.628929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39f49ba060b4bea879adba92a34809c', 'request_id': 'req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.638734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39f49ba060b4bea879adba92a34809c', 'request_id': 'req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.648487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39f49ba060b4bea879adba92a34809c', 'request_id': 'req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.662588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] The token cache is being invalidate because project a39f49ba060b4bea879adba92a34809c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:41.663406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.664647 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e86dfe9f-47c6-4095-bd45-7eb8f9835ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:41.671348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1272/2507] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:41 2022] DELETE /identity/v3/projects/a39f49ba060b4bea879adba92a34809c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:41.673737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:41.674047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:41.674317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:41.695073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:41.697007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:41.791640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1236/2508] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:41.799792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:41.800081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:41.800406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:41.823189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:41.824899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:41.838143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1273/2509] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:41.842189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:41.859369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe7077-d72c-47a8-96fb-368263dfdad3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:41.874576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:41.879418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:41.880064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:41.880659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:41.881784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:41.882267 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:41.884942 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:41.908197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:41.908993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1237/2510] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:28:41.919691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4449e6-ec38-4c24-a127-5638d4d4a681 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:41.929119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:41.931823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:41.938476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:41.938675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:41.940704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:41.943730 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:41.991008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1274/2511] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:41 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:41.992949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.017418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d428ff-5347-4861-9e09-7b1eabe88d25 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.138999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.140853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:42.186739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.187910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.188284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.189335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.190113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.193188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.239212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1238/2512] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:41 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:42.242535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.260893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c177ef8-52ee-4892-a36b-b4016d5f1d85 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.286411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.287658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.288476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.289009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.290011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.293235 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.298656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a87d-52f7-47c8-ad1c-76686626a678 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.305279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.306206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.308649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.309336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.316179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:42.339778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1239/2513] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:42.340015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.340877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.342079 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.353653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1275/2514] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:42.356399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ae991-ce91-4ef2-b094-8e33ec789ce3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.365690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.369218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.369661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.370856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.372202 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.381178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65792e9-3909-4038-8382-ba12a841fccc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.408646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1240/2515] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:42.412772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.434203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8afd-aba4-4ffb-af5e-c619a6c1a1c0 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.448973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.452677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.453108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.455806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] PATH_INFO: `/v3/projects/1067d8cac01a47b8ac47fce5cb2b5c23/users/c92f5172848c467cb6bd8aabb22ada88/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.491816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1067d8cac01a47b8ac47fce5cb2b5c23, user_id=c92f5172848c467cb6bd8aabb22ada88, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.498195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a1f4e-fed9-4366-a235-a0bbd8b8e5dd admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.527754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.528283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.528818 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.533283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1276/2516] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:42 2022] PUT /identity/v3/projects/1067d8cac01a47b8ac47fce5cb2b5c23/users/c92f5172848c467cb6bd8aabb22ada88/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:42.546257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.546596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.546864 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.572147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] MFA Rules not processed for user `4b84b7192b13421286d86ba3a43d0475`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:42.573928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:42.605545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] MFA Rules not processed for user `c92f5172848c467cb6bd8aabb22ada88`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:42.607274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:42.744315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1277/2517] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:42.747996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.756634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1241/2518] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:28:42.761074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.763533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:42.852383 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f1d6f-8178-44e6-9983-41d359eda4ce None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.860738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.861942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.862462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.862966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.863810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.865959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.876612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405901f-c43c-43ee-915b-8d88ce21a232 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:42.882174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:42.883299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:42.883773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:42.884155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:42.884924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:42.887293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:42.907906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1278/2519] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:42.911929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:42.936804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1242/2520] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:42.940906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.010503 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f41f-f774-4f74-b9ae-9571825996e4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.029739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a5162-ddea-452b-9812-66f8bba7b469 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.035657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.036742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.037283 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.037662 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.038460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.040525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.067095 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1243/2521] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:43.071136 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.085769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf52bd5-848d-46f9-8955-a5b921bf7ba2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.092731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.094564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.095184 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.095603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.096577 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.099050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.100112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.100844 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.101125 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.101395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.102089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.104031 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.116290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1244/2522] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:43.120450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.133683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1279/2523] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:42 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:43.136975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d3745-aff3-41dd-80e5-2dc658a1200d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.137998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.143901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.144907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.145445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.145900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] PATH_INFO: `/v3/projects/7bb154c009f64a7eb99123226bb8cb4f/users/62854276e03e4f91b2589c906c9c5240/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.162554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683520d-06fb-4e62-b04e-b9643dd27177 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.172146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb154c009f64a7eb99123226bb8cb4f, user_id=62854276e03e4f91b2589c906c9c5240, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.173059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.173984 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.174494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.175050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.175185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.176111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.177956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.193851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1245/2524] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:43 2022] PUT /identity/v3/projects/7bb154c009f64a7eb99123226bb8cb4f/users/62854276e03e4f91b2589c906c9c5240/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:43.196309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1280/2525] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:43.199672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.201281 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.221239 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a977-2dce-4add-9c8e-6aba893382f4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.305490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.306398 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.306684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.307615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] PATH_INFO: `/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7/users/32aa026f273f48eca59e31751bb0955e/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.311265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc1347-881e-4c56-bd6a-96893ba29c7a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.318367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.319586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.319867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.320143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.321702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.322924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.329296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72458afae3bd45e7a06d25ddb47fe9f7, user_id=32aa026f273f48eca59e31751bb0955e, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.332144 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.342098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1246/2526] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:43.347185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.352728 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1281/2527] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:43 2022] PUT /identity/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7/users/32aa026f273f48eca59e31751bb0955e/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:43.356809 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.359597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587ef8-64b6-4448-8085-ad29b113b05d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.444227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.445252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.445768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.446203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] PATH_INFO: `/v3/projects/7bb154c009f64a7eb99123226bb8cb4f/users/62854276e03e4f91b2589c906c9c5240/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.448740 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb154c009f64a7eb99123226bb8cb4f, user_id=62854276e03e4f91b2589c906c9c5240, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.449183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd394d4-9dab-4706-95b8-169d1ad29a73 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.452053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.463976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.464714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1247/2528] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:43 2022] PUT /identity/v3/projects/7bb154c009f64a7eb99123226bb8cb4f/users/62854276e03e4f91b2589c906c9c5240/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:43.465189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.465900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.466253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.467201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.469039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b36bf-0a6e-48bb-b5fc-c4ad905c0cf7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.480978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1282/2529] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:43.485056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.568368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df20658-21d1-4c42-b601-245f4a4478b1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.573601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.574727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.575007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.575289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] PATH_INFO: `/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7/users/32aa026f273f48eca59e31751bb0955e/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.577645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72458afae3bd45e7a06d25ddb47fe9f7, user_id=32aa026f273f48eca59e31751bb0955e, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.580996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.607634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1248/2530] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:43 2022] PUT /identity/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7/users/32aa026f273f48eca59e31751bb0955e/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:43.618137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.618843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.618843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.674512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] MFA Rules not processed for user `32aa026f273f48eca59e31751bb0955e`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:43.675757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:43.817379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1283/2531] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:43.831252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:43.833032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:43.923145 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaabdb-97d0-48e8-b211-b899ec48c72c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:43.929413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:43.930852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:43.931609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:43.932101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:43.950618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:43.952520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40b15e-41e2-4ea8-b9fc-c9d23e6ed00b service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:43.970570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1249/2532] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:44.621441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:44.796915 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ee572-e812-488a-a0a4-7970e403d458 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:44.811848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:44.812883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:44.813172 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:44.813454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] PATH_INFO: `/v3/users/c92f5172848c467cb6bd8aabb22ada88` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:44.814693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92f5172848c467cb6bd8aabb22ada88)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:44.816666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:44.919715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92f5172848c467cb6bd8aabb22ada88'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:44.938941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92f5172848c467cb6bd8aabb22ada88'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:44.939746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92f5172848c467cb6bd8aabb22ada88'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:44.940494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92f5172848c467cb6bd8aabb22ada88'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:44.968155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92f5172848c467cb6bd8aabb22ada88'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.005689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1284/2533] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:44 2022] DELETE /identity/v3/users/c92f5172848c467cb6bd8aabb22ada88 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:45.013507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:45.014117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:45.014117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:45.049260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:45.051917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.194237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:45.196166 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.207097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1250/2534] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:45.221727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29262d30-c975-4e0f-a8c3-4aa45f9d717d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:45.229833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:45.230739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:45.231016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:45.231343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] PATH_INFO: `/v3/users/59f39e7407514661a5542e9ef8cde7ac` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:45.232634 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f39e7407514661a5542e9ef8cde7ac)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:45.234650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:45.305907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:45.308183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.327865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f39e7407514661a5542e9ef8cde7ac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.344063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f39e7407514661a5542e9ef8cde7ac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.345037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f39e7407514661a5542e9ef8cde7ac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.345562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f39e7407514661a5542e9ef8cde7ac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.353461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-1fa58e10-e08d-4d0c-b275-b73e2cf3a739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f39e7407514661a5542e9ef8cde7ac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:45.387773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1285/2535] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:45 2022] DELETE /identity/v3/users/59f39e7407514661a5542e9ef8cde7ac => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:45.392304 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:45.392704 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:45.393014 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:45.434318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cd0e6-a46d-4b78-ac1e-30d191192217 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:45.440815 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:45.442047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.539866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:45.542018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:45.543468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:45.548432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:45.551482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.616928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1286/2536] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:45.630131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:45.632654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:45.656212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd94d-82c6-4786-a11f-b6444ed3b7bf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:45.660673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:45.661333 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:45.661611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:45.661883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:45.681695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:45.683141 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:45.714659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1287/2537] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:45.717808 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:45.719432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:45.741315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1251/2538] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:46.718482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:46.720579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:46.742422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:46.744956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:46.746631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574ead8-1f01-4a0a-9576-dc7bc98855b3 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:46.761257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:46.762496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:46.764193 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:46.764708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] PATH_INFO: `/v3/projects/1067d8cac01a47b8ac47fce5cb2b5c23` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:46.766378 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1067d8cac01a47b8ac47fce5cb2b5c23)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:46.767902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f595a4d-d836-4186-91e3-206bd0a36145 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:46.768900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:46.787083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:46.787977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:46.788506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:46.790251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] PATH_INFO: `/v3/projects/6259e1e9b8cf4123a213c7269ba80154` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:46.792971 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6259e1e9b8cf4123a213c7269ba80154)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:46.796454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:46.920205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1067d8cac01a47b8ac47fce5cb2b5c23', 'request_id': 'req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.927352 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6259e1e9b8cf4123a213c7269ba80154', 'request_id': 'req-8ca53efb-abc0-46ce-8ede-c244daa1abf9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.933999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1067d8cac01a47b8ac47fce5cb2b5c23', 'request_id': 'req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.941697 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6259e1e9b8cf4123a213c7269ba80154', 'request_id': 'req-8ca53efb-abc0-46ce-8ede-c244daa1abf9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.963465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6259e1e9b8cf4123a213c7269ba80154', 'request_id': 'req-8ca53efb-abc0-46ce-8ede-c244daa1abf9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.985590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] The token cache is being invalidate because project 6259e1e9b8cf4123a213c7269ba80154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:46.985885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.986869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1067d8cac01a47b8ac47fce5cb2b5c23', 'request_id': 'req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:46.992576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.003557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1252/2539] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:46 2022] DELETE /identity/v3/projects/6259e1e9b8cf4123a213c7269ba80154 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:47.007615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:47.016536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] The token cache is being invalidate because project 1067d8cac01a47b8ac47fce5cb2b5c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:47.016945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.018645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.021190 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:47.022045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1288/2540] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:46 2022] DELETE /identity/v3/projects/1067d8cac01a47b8ac47fce5cb2b5c23 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:47.029285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:47.031158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:47.202941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53efb-abc0-46ce-8ede-c244daa1abf9 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:47.215521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c0a27-9ce1-4033-8aac-3f7f7ad65c1b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:47.220854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:47.223096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:47.224153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:47.224398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:47.224741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:47.224881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:47.225031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] PATH_INFO: `/v3/users/af916674c9e14564b4cbd2ea5a11f04f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:47.225783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] PATH_INFO: `/v3/users/32aa026f273f48eca59e31751bb0955e` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:47.231058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af916674c9e14564b4cbd2ea5a11f04f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:47.232994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32aa026f273f48eca59e31751bb0955e)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:47.235749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:47.236938 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:47.431758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32aa026f273f48eca59e31751bb0955e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.469966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af916674c9e14564b4cbd2ea5a11f04f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.516401 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32aa026f273f48eca59e31751bb0955e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.517324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af916674c9e14564b4cbd2ea5a11f04f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.517651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32aa026f273f48eca59e31751bb0955e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.520735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32aa026f273f48eca59e31751bb0955e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.540965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af916674c9e14564b4cbd2ea5a11f04f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.542622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af916674c9e14564b4cbd2ea5a11f04f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.563625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32aa026f273f48eca59e31751bb0955e'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.574767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af916674c9e14564b4cbd2ea5a11f04f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:47.585119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1253/2541] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:47 2022] DELETE /identity/v3/users/af916674c9e14564b4cbd2ea5a11f04f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:47.599197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1289/2542] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:47 2022] DELETE /identity/v3/users/32aa026f273f48eca59e31751bb0955e => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:48.505646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:48.510438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:48.730308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c4dc-6bfe-4fd2-a750-f1efc4211120 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:48.824493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:48.827075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:48.975153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:48.976557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:48.977043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:48.977490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] PATH_INFO: `/v3/projects/aa6300bbaa754a6599079cff2567b7c8` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:48.979244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6300bbaa754a6599079cff2567b7c8)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:48.982317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.010954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb11f-111d-467c-92ae-c6cb27d9db94 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.025897 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.026806 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.030164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.030764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] PATH_INFO: `/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.033083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72458afae3bd45e7a06d25ddb47fe9f7)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.038458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.067035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6300bbaa754a6599079cff2567b7c8', 'request_id': 'req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.081303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6300bbaa754a6599079cff2567b7c8', 'request_id': 'req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.113977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6300bbaa754a6599079cff2567b7c8', 'request_id': 'req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.169634 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] The token cache is being invalidate because project aa6300bbaa754a6599079cff2567b7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:49.169976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.176035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d62dea9e-a1fa-4c94-ba1a-1f5f7f2efab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.176925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72458afae3bd45e7a06d25ddb47fe9f7', 'request_id': 'req-4bba8bf2-5623-484e-a984-1ae8b041321d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.180330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1254/2543] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:48 2022] DELETE /identity/v3/projects/aa6300bbaa754a6599079cff2567b7c8 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:49.186658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e527b-7fb0-482e-8831-1e462ca0d89c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.186985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e527b-7fb0-482e-8831-1e462ca0d89c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.187238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e527b-7fb0-482e-8831-1e462ca0d89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.191831 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72458afae3bd45e7a06d25ddb47fe9f7', 'request_id': 'req-4bba8bf2-5623-484e-a984-1ae8b041321d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.213743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72458afae3bd45e7a06d25ddb47fe9f7', 'request_id': 'req-4bba8bf2-5623-484e-a984-1ae8b041321d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.239557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] The token cache is being invalidate because project 72458afae3bd45e7a06d25ddb47fe9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:49.240046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.241551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-4bba8bf2-5623-484e-a984-1ae8b041321d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:49.244313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ae9e527b-7fb0-482e-8831-1e462ca0d89c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:49.245368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1290/2544] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:48 2022] DELETE /identity/v3/projects/72458afae3bd45e7a06d25ddb47fe9f7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:49.248480 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ae9e527b-7fb0-482e-8831-1e462ca0d89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:49.250595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3897d-d797-44cc-b379-53d4e600b2d8 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.251034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3897d-d797-44cc-b379-53d4e600b2d8 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.251436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3897d-d797-44cc-b379-53d4e600b2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.309002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-77b3897d-d797-44cc-b379-53d4e600b2d8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:49.312440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-77b3897d-d797-44cc-b379-53d4e600b2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:49.436140 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1255/2545] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:49.438265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.438688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.439137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.459258 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1291/2546] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:49.461206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.461510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.461793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.489504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:49.490920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:49.492406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:49.493746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:49.503612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1292/2547] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:49.509142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1256/2548] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:49.509627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.512420 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.520344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75a423-ac81-4f57-99c2-6cffb069aaff None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.521963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d5a97-3fc0-4ec3-ad9f-10bd51c5533d None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.525469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.526448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.526448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.526687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.527460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:49.527811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.529874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.536873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.537803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.538254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.538694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.539601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:49.540472 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.542296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.556803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1293/2549] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:28:49.557746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.573594 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1804-0f54-4f7c-9e9c-e25db5a23dde admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.574013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1257/2550] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:28:49.578979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.596367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ab137-eff4-411b-a5b3-4382f6373f60 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.596823 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.597522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.597826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.600837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.601581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.607851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.614596 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.615600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.616081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.616631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.617536 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.619710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.689043 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1258/2551] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:49.690312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1294/2552] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:49.691695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.693250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.792342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3a0b6-5ce7-4a39-b496-f006b1e06e38 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.838337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895605-6151-4bf9-8bc9-d876628ecb60 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.850851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.851821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.852278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.852501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.853245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.856648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.866640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.868277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.869154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.870088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.871116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.874264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.920076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1295/2554] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:49.920727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1259/2554] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:49 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:49.921434 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.921702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:49.923805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:49.940707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3cabb-e8da-4fd5-bd6a-461e9120e112 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.942249 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434246-1237-498a-bc52-f725f8d74430 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:49.947373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.948842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.949489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.950012 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] PATH_INFO: `/v3/users/6a41fc2b6e1c4032bfa408eaf36887ee` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.952177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a41fc2b6e1c4032bfa408eaf36887ee)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.955506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:49.956424 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:49.957339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:49.957733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:49.958532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:49.960167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.962504 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:49.987250 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1260/2555] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:49.992620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.012058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7a32f-f573-43c8-8d3d-f296fb1cd41b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.019243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.020015 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.020357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.020639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.021626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.024234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.049535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1261/2556] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:50.054351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.062369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a41fc2b6e1c4032bfa408eaf36887ee'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.067393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea6f5b-3401-4139-a160-f46f7652959b admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.073734 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.075019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.075450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.075973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.077044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.079093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.082483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a41fc2b6e1c4032bfa408eaf36887ee'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.083959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a41fc2b6e1c4032bfa408eaf36887ee'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.085947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a41fc2b6e1c4032bfa408eaf36887ee'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.094474 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1262/2557] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:50.095389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a41fc2b6e1c4032bfa408eaf36887ee'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.095902 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.097876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.114928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899ca8-8584-4d16-b2d0-dd91961a7683 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.118493 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1296/2558] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:49 2022] DELETE /identity/v3/users/6a41fc2b6e1c4032bfa408eaf36887ee => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:50.119685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.124955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.197924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.198753 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.199041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.199313 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.200054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.202454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.217273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1263/2559] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:50.220684 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.236556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd217c-cd8e-4198-90db-f8eb9bc77ca0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.237767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee4747-7db7-4d8e-bb98-1bda44984e11 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.243916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.245309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.245605 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.245881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] PATH_INFO: `/v3/projects/0663b65706374bd38072153465b6f130/users/7b5a1d33ed364cfbad70668d5ee2cb4c/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.251068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.254167 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.254457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.256156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] PATH_INFO: `/v3/projects/873aaccbd05a43939276cffc1b7d8b2a/users/26ef4f18d9f84f59aac68b44739381fe/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.268018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0663b65706374bd38072153465b6f130, user_id=7b5a1d33ed364cfbad70668d5ee2cb4c, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.271807 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d6b1b-a0b1-4c65-9aae-b669872aa645 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.282464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=873aaccbd05a43939276cffc1b7d8b2a, user_id=26ef4f18d9f84f59aac68b44739381fe, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.286199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4572d-1062-485d-93a9-7ab99844c30d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.300501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1264/2560] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:50 2022] PUT /identity/v3/projects/0663b65706374bd38072153465b6f130/users/7b5a1d33ed364cfbad70668d5ee2cb4c/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:50.306809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.307285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.307765 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.353974 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1297/2561] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:50 2022] PUT /identity/v3/projects/873aaccbd05a43939276cffc1b7d8b2a/users/26ef4f18d9f84f59aac68b44739381fe/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:50.386620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] MFA Rules not processed for user `7b5a1d33ed364cfbad70668d5ee2cb4c`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:50.388209 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.414090 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.414411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.414694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.456287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] MFA Rules not processed for user `26ef4f18d9f84f59aac68b44739381fe`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:50.458066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.556843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1298/2562] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:50.557830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.559244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.609930 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1265/2563] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:50.610848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.617331 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.672553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512c351-a35f-4879-a84c-3c58c41b03ac None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.687541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.689146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.689444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.689743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] PATH_INFO: `/v3/projects/9d233fdd15ef41ab9e5cf40029fe5ac4` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.713077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d233fdd15ef41ab9e5cf40029fe5ac4)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.716110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.741950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebea8-9f85-4ffb-b5f5-5a3f3e8f72a7 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.746311 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89fc09941e0b4aa29bbfe6d41e154f7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.746980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.747476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.747758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.765151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.766636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.778208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1266/2564] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:50.778900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d233fdd15ef41ab9e5cf40029fe5ac4', 'request_id': 'req-8bcdb3f9-539c-4365-b110-58596b88b284'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.782301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.793670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae5c46-cb6f-4c37-8ae8-0c0ef1eaa977 service glance] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:50.795248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d233fdd15ef41ab9e5cf40029fe5ac4', 'request_id': 'req-8bcdb3f9-539c-4365-b110-58596b88b284'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.814667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d233fdd15ef41ab9e5cf40029fe5ac4', 'request_id': 'req-8bcdb3f9-539c-4365-b110-58596b88b284'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.835157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] The token cache is being invalidate because project 9d233fdd15ef41ab9e5cf40029fe5ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:50.835768 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.838291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8bcdb3f9-539c-4365-b110-58596b88b284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:50.845344 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1299/2565] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:50 2022] DELETE /identity/v3/projects/9d233fdd15ef41ab9e5cf40029fe5ac4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:50.848189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9bf9-4b4b-42b9-9670-0823cd0435cf None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.848969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9bf9-4b4b-42b9-9670-0823cd0435cf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.850081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9bf9-4b4b-42b9-9670-0823cd0435cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.887129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-be7e9bf9-4b4b-42b9-9670-0823cd0435cf None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:50.889358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-be7e9bf9-4b4b-42b9-9670-0823cd0435cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.895759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:50.898178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.898652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.899088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.900307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:50.902638 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:50.904677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1300/2566] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:50.913963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:50.914257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:50.914542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:50.951508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:50.951951 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1267/2567] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:50.953107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:50.955990 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:50.968609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:51.053342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89063cd2-2c3f-493b-836c-e6553fd19aee admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.070258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.071655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.072103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.072567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.073387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.076327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.096810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1301/2568] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:51.101524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.104011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1268/2569] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:50 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:51.104931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.116645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe7d-5490-42d8-965d-22f0adbc1a17 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.120684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1b1f0-67cb-47ac-823f-ef5163b5c429 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.122433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.123622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.123986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.124354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.126428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.126769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.127697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.128080 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.128515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.128721 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.129571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:51.129978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.131693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.146290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1269/2570] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:51.147897 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.160853 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192aec9-8051-4f41-80a9-5ffdc7d95a4f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.165414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1302/2571] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:28:51.169428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.170729 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.175163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.175163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.175163 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] PATH_INFO: `/v3/projects/fe69ab92ba464d44987a911ef46d9bd9/users/d7d09cb3c6fc44e089bca3ddcee1ebc9/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.200966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632642cc-3163-43bc-9e29-b987b6a7270d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.207432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.207707 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe69ab92ba464d44987a911ef46d9bd9, user_id=d7d09cb3c6fc44e089bca3ddcee1ebc9, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.209236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.209602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.209952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.210168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.210783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.213397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.235521 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1270/2572] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:51 2022] PUT /identity/v3/projects/fe69ab92ba464d44987a911ef46d9bd9/users/d7d09cb3c6fc44e089bca3ddcee1ebc9/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:51.239245 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.268751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1303/2573] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:51 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:51.287702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.408985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5420b3d-cedd-47ce-872e-2c250d324779 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.411279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769497f6-9303-466d-91b5-9b91781b06dd admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.421841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.422886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.423389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.423671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.423962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.424898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.425904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.426386 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.426675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.428779 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.430471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.430860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.461450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1271/2574] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:28:51.466417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.478882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1304/2575] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:51 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:51.480528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbfd5e-40b3-4c58-9fd2-e92b88687788 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.483836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.494258 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.495626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.496149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.496822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] PATH_INFO: `/v3/projects/fe69ab92ba464d44987a911ef46d9bd9/users/d7d09cb3c6fc44e089bca3ddcee1ebc9/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.499490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe69ab92ba464d44987a911ef46d9bd9, user_id=d7d09cb3c6fc44e089bca3ddcee1ebc9, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.502686 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037dbc8-f4c3-4b1d-8add-f20caf5bcba6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.515030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477122-efe9-4202-b8e8-9ab94c9ab868 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.539410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1272/2576] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:51 2022] PUT /identity/v3/projects/fe69ab92ba464d44987a911ef46d9bd9/users/d7d09cb3c6fc44e089bca3ddcee1ebc9/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:28:51.555121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.555490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.556000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.629821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.631784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.634663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.635063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.637621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.638520 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] MFA Rules not processed for user `d7d09cb3c6fc44e089bca3ddcee1ebc9`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:51.639688 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:51.640703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.670008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1305/2577] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:51.674560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.697412 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da75f49-3fc2-4267-8312-7475263cbc89 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.714478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.715265 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.715755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.716222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.717284 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.719929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.745836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1306/2578] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:51.751078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.763453 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e39be-c9f8-4722-81f5-e45e3c44aa67 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.764922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1273/2579] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:51.771660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.772586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.772867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.773639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] PATH_INFO: `/v3/projects/3c43d22712ef48ada5bb2d940efc3372/users/294d1c4723964b3cb145a74c3c68f91d/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.779427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:51.780967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:51.793365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c43d22712ef48ada5bb2d940efc3372, user_id=294d1c4723964b3cb145a74c3c68f91d, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:51.797539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9c9e7-c051-4723-be09-a749048917ef admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:51.817058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1307/2580] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:51 2022] PUT /identity/v3/projects/3c43d22712ef48ada5bb2d940efc3372/users/294d1c4723964b3cb145a74c3c68f91d/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:51.818638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8c75-5995-49fb-b25e-2ab14d00167e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.818976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8c75-5995-49fb-b25e-2ab14d00167e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.819313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8c75-5995-49fb-b25e-2ab14d00167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.859043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-69db8c75-5995-49fb-b25e-2ab14d00167e None None] MFA Rules not processed for user `62854276e03e4f91b2589c906c9c5240`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:51.860524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-69db8c75-5995-49fb-b25e-2ab14d00167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:51.875826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42153b05-9eb5-4eb6-9e2c-5f3b75531455 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:51.936409 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1308/2581] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:51.938357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.938862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.939343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.947338 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:51.948009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:51.948446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:51.948724 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:51.981560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] MFA Rules not processed for user `294d1c4723964b3cb145a74c3c68f91d`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:51.982771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:52.021611 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.023303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.033219 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1274/2582] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:28:52.034031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.035361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:52.067646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1309/2583] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:52.081181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.085614 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:52.100795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e508a16-28c8-46b8-9108-d2d7f098f4a6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.105315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.106013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.106306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.106625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.117208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.118618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.129692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1310/2584] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:52.160053 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82bc9b-d8d8-47aa-b579-9254c71a123e service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.166394 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.167056 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.167351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.168300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.170523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:52.269355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.270954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.283609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1275/2585] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:52.463017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.487044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5d883-9888-419c-aa74-8e3ebf15eb7d service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.497742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d09cb3c6fc44e089bca3ddcee1ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe69ab92ba464d44987a911ef46d9bd9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.498641 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.498926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.499223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.501005 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:52.501318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.507814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.541098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1311/2586] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:28:52.545769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.558914 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12aa39-e581-422d-b830-9251b4491e64 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.565040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d09cb3c6fc44e089bca3ddcee1ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe69ab92ba464d44987a911ef46d9bd9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.566148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.566643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.567370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.568310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.571814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.617018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1276/2587] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:52 2022] POST /identity/v3/projects => generated 353 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:52.617152 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.630467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af5655-0061-4281-af8f-6b90f6d396b0 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.742405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d09cb3c6fc44e089bca3ddcee1ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe69ab92ba464d44987a911ef46d9bd9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.742880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.743432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.743627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.745222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:52.745878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.750106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.774130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1312/2588] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:28:52.819589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:52.835543 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d0f6c-17f9-4ff9-a041-92c0bcce73ef tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:52.840746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7d09cb3c6fc44e089bca3ddcee1ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe69ab92ba464d44987a911ef46d9bd9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:52.841910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:52.842135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:52.842693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] PATH_INFO: `/v3/projects/5af2762d21ce4be6a9ed178e50b7738c` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:52.844173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5af2762d21ce4be6a9ed178e50b7738c)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:52.846514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:52.901337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af2762d21ce4be6a9ed178e50b7738c', 'request_id': 'req-11601f18-ad96-4036-b787-c14186601a32'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:52.909007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af2762d21ce4be6a9ed178e50b7738c', 'request_id': 'req-11601f18-ad96-4036-b787-c14186601a32'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:52.917598 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af2762d21ce4be6a9ed178e50b7738c', 'request_id': 'req-11601f18-ad96-4036-b787-c14186601a32'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:52.928849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] The token cache is being invalidate because project 5af2762d21ce4be6a9ed178e50b7738c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:52.929200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:52.929998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-11601f18-ad96-4036-b787-c14186601a32 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:52.931561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1277/2589] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:52 2022] DELETE /identity/v3/projects/5af2762d21ce4be6a9ed178e50b7738c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:53.187912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:53.190469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:53.276695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275df41-2af0-4ea4-9b2e-5bdcc4ed5dd7 tempest-VolumeQuotaClassesTest-1183007740 tempest-VolumeQuotaClassesTest-1183007740-project-admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:53.348888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:53.349926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:53.350206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:53.350478 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] PATH_INFO: `/v3/users/7b5a1d33ed364cfbad70668d5ee2cb4c` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:53.351673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5a1d33ed364cfbad70668d5ee2cb4c)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:53.354139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:53.417609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5a1d33ed364cfbad70668d5ee2cb4c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:53.437477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5a1d33ed364cfbad70668d5ee2cb4c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:53.438291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5a1d33ed364cfbad70668d5ee2cb4c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:53.439057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5a1d33ed364cfbad70668d5ee2cb4c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:53.452986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5a1d33ed364cfbad70668d5ee2cb4c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:53.470752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1313/2590] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:53 2022] DELETE /identity/v3/users/7b5a1d33ed364cfbad70668d5ee2cb4c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:53.475764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:53.476020 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:53.476487 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:53.500295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:53.501573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:53.585863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1278/2591] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:53.594651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:53.594651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:53.698291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2eeb-9b9d-48f2-b8bd-75937a467dc6 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:53.704064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:53.704825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:53.705165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:53.705500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:53.719155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:53.720735 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:53.733452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1314/2592] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:54.203804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:54.206248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:54.224052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ada02-1e94-4363-8eb5-ecc624308c25 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:54.243790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:54.247432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:54.248003 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:54.248493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] PATH_INFO: `/v3/projects/0663b65706374bd38072153465b6f130` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:54.252242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0663b65706374bd38072153465b6f130)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:54.256270 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:54.346483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0663b65706374bd38072153465b6f130', 'request_id': 'req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.360712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0663b65706374bd38072153465b6f130', 'request_id': 'req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.380046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0663b65706374bd38072153465b6f130', 'request_id': 'req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.410074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] The token cache is being invalidate because project 0663b65706374bd38072153465b6f130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:54.410467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.411790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.414147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1279/2593] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:54 2022] DELETE /identity/v3/projects/0663b65706374bd38072153465b6f130 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:54.418476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:54.430362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1eb7e-c7cc-47f9-902f-410785eb5c9f service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:54.501294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:54.502199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:54.502471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:54.502740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] PATH_INFO: `/v3/users/d7d09cb3c6fc44e089bca3ddcee1ebc9` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:54.505158 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d09cb3c6fc44e089bca3ddcee1ebc9)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:54.506088 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:54.559399 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d09cb3c6fc44e089bca3ddcee1ebc9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.573925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d09cb3c6fc44e089bca3ddcee1ebc9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.574979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d09cb3c6fc44e089bca3ddcee1ebc9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.576308 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d09cb3c6fc44e089bca3ddcee1ebc9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.584232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-a6716ae6-e4ea-4ea6-80cc-a047e065b08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d09cb3c6fc44e089bca3ddcee1ebc9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:54.599207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1315/2594] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:28:54 2022] DELETE /identity/v3/users/d7d09cb3c6fc44e089bca3ddcee1ebc9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:54.949287 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:54.951089 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:54.965701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a44a2-aaf6-45a6-8ef9-fc8fa24bdf0e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.033887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.035405 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.035891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.036458 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] PATH_INFO: `/v3/projects/fe69ab92ba464d44987a911ef46d9bd9` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.037900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe69ab92ba464d44987a911ef46d9bd9)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.040168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.097976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe69ab92ba464d44987a911ef46d9bd9', 'request_id': 'req-c9680749-98e7-460a-b459-88eca1066a3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:55.104162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe69ab92ba464d44987a911ef46d9bd9', 'request_id': 'req-c9680749-98e7-460a-b459-88eca1066a3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:55.113733 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe69ab92ba464d44987a911ef46d9bd9', 'request_id': 'req-c9680749-98e7-460a-b459-88eca1066a3c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:55.124115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] The token cache is being invalidate because project fe69ab92ba464d44987a911ef46d9bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:28:55.124468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:55.125121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c9680749-98e7-460a-b459-88eca1066a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:28:55.126964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1280/2595] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:28:54 2022] DELETE /identity/v3/projects/fe69ab92ba464d44987a911ef46d9bd9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:55.132805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.132805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.132805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.160935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:55.162246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:55.297588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1316/2596] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:55.306225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.306766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.306872 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.330690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:55.330690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:55.339489 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1281/2597] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:55.345138 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.364746 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caccb21-fdbb-4a4c-8e58-19424960f630 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.371400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.372310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.372689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.373010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.374323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:28:55.374527 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.379862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.397185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1317/2598] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:28:55.398398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.407977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1090f-887b-4744-a5ef-74d173cceda4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.412783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.413464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.413748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.414024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.414728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.416687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.448547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1282/2599] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:55.452847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.465325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff14042-f514-4149-8385-701b7342c59a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.528968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.529651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.529932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.530207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.530908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.532955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.553393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1318/2600] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:55.557719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.568505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b111be-1aed-4ac0-9a6e-af7d9383b00c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.573666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.574336 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.574616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.574885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.575848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.577764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.588950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1283/2601] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:55.593462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.603906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f7f39-f114-4cdd-b636-05bfdf11b740 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.608825 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.609487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.609760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.610022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.610765 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.612368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252d7ce-6e76-4d26-b3bf-b1441d4a0c84 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.622957 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1319/2602] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:55.627157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.637225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15278dec-51c5-4538-9078-d29b68b6e98c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.642035 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.643252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.643252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.643460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] PATH_INFO: `/v3/projects/7b745f8b423a4653a4dbc2cf86f51212/users/9bc597708a5f4cb6bcb27abde173c7a6/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.659187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b745f8b423a4653a4dbc2cf86f51212, user_id=9bc597708a5f4cb6bcb27abde173c7a6, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.661865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.678778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1284/2603] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:55 2022] PUT /identity/v3/projects/7b745f8b423a4653a4dbc2cf86f51212/users/9bc597708a5f4cb6bcb27abde173c7a6/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:55.689801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.690335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.690673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.722235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] MFA Rules not processed for user `9bc597708a5f4cb6bcb27abde173c7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:55.723435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:55.823160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1320/2604] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:55.828396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.904873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e55a-4f2b-4fef-8ece-9ce6f7b1d23d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:55.911581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:55.920014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:55.920014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:55.920014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:55.920014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:55.920014 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:55.946636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1285/2605] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:28:55.951395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:55.962985 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f037be-b777-4258-8fe5-5ff6eef440dc None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.024121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.024908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.025276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.025616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.026394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.029918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.059521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1321/2606] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:28:55 2022] POST /identity/v3/users => generated 302 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:28:56.063647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.077517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0589-640f-4646-89ba-9e5d1602ea9a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.082476 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.083175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.083459 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.083738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.084513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.086137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.103909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1286/2607] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:56.105245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.116060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4034c-7303-4f59-977c-562d0ab10034 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.121535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.122708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.122990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.123262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] PATH_INFO: `/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc/users/c9d83ef0c51b419ba240cbea5ae633a5/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.140121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bac0e2f07b444a2826a42e8ca1942dc, user_id=c9d83ef0c51b419ba240cbea5ae633a5, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.147226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.158390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1322/2608] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:56 2022] PUT /identity/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc/users/c9d83ef0c51b419ba240cbea5ae633a5/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:56.162437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.174498 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848340f-d555-48b1-8a34-6fa99bcc8806 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.231859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.232627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.233436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.233436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.233901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.235546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a62fb-6c4b-4aa7-8d36-eeee77d6e591 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.247364 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1287/2609] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:28:56.251512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.269676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2043f-ba6c-4961-b155-49a6dc9e88b5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.276151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.277040 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.277323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.277601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] PATH_INFO: `/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc/users/c9d83ef0c51b419ba240cbea5ae633a5/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.285929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bac0e2f07b444a2826a42e8ca1942dc, user_id=c9d83ef0c51b419ba240cbea5ae633a5, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.285929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.298710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1323/2610] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:28:56 2022] PUT /identity/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc/users/c9d83ef0c51b419ba240cbea5ae633a5/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:28:56.306256 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.306799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.306799 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.338346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] MFA Rules not processed for user `c9d83ef0c51b419ba240cbea5ae633a5`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:28:56.339890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:56.462074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1288/2611] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:28:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:28:56.471101 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.472925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:28:56.576436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92b179-61cb-469c-bceb-4baa1268b8f1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.580317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.581109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.581883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.581883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.593275 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.595062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.605446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1324/2612] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:28:56.666530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:28:56.688852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11448a8-1bb0-42de-a757-687ca6f4ea29 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:28:56.697296 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:28:56.697961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:28:56.698237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:28:56.698512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:28:56.781672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:28:56.783323 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ee45a-33b6-4479-8380-cadbbf6f1c71 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:28:56.792999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1289/2613] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:28:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:01.246147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:01.312738 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b6ec-f2ba-4aa4-ab97-7785af0363e7 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:01.317402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:01.318286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:01.318569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:01.318880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] PATH_INFO: `/v3/users/9bc597708a5f4cb6bcb27abde173c7a6` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:01.320155 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc597708a5f4cb6bcb27abde173c7a6)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:01.322489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:01.373389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc597708a5f4cb6bcb27abde173c7a6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:01.385542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc597708a5f4cb6bcb27abde173c7a6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:01.386392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc597708a5f4cb6bcb27abde173c7a6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:01.387050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc597708a5f4cb6bcb27abde173c7a6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:01.393954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc597708a5f4cb6bcb27abde173c7a6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:01.409441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1325/2614] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:01 2022] DELETE /identity/v3/users/9bc597708a5f4cb6bcb27abde173c7a6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:01.414317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:01.414800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:01.414881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:01.434558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:01.435754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:01.500637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1290/2615] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:01.510788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:01.512355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:01.579607 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f436712-0dc0-454a-a99c-a3b2a2d1847f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:01.586007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:01.586906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:01.587229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:01.587487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:01.601783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:01.603218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:01.613340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1326/2616] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:01.899842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:01.901732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:01.932440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e2a8-ed82-466c-acbd-9c9ac2233ca6 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:01.937088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:01.937960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:01.938243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:01.938637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] PATH_INFO: `/v3/projects/7b745f8b423a4653a4dbc2cf86f51212` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:01.939835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b745f8b423a4653a4dbc2cf86f51212)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:01.941769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:01.995571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b745f8b423a4653a4dbc2cf86f51212', 'request_id': 'req-43e316b1-7264-4b42-bd99-1a537fbc36fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.001964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b745f8b423a4653a4dbc2cf86f51212', 'request_id': 'req-43e316b1-7264-4b42-bd99-1a537fbc36fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.010107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b745f8b423a4653a4dbc2cf86f51212', 'request_id': 'req-43e316b1-7264-4b42-bd99-1a537fbc36fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.020862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] The token cache is being invalidate because project 7b745f8b423a4653a4dbc2cf86f51212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:02.021148 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.021793 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.023341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1291/2617] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:01 2022] DELETE /identity/v3/projects/7b745f8b423a4653a4dbc2cf86f51212 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:02.027295 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.038111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef248-28bd-494a-ace8-1fb42addd68a service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.094484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.095360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.095637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.095924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] PATH_INFO: `/v3/users/c9d83ef0c51b419ba240cbea5ae633a5` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.097186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d83ef0c51b419ba240cbea5ae633a5)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.099105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.152039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d83ef0c51b419ba240cbea5ae633a5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.165128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d83ef0c51b419ba240cbea5ae633a5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.165967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d83ef0c51b419ba240cbea5ae633a5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.166611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d83ef0c51b419ba240cbea5ae633a5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.173578 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-376b475e-9082-4c9b-9dd5-8a3c3cd0d32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d83ef0c51b419ba240cbea5ae633a5'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.190011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1327/2618] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:02 2022] DELETE /identity/v3/users/c9d83ef0c51b419ba240cbea5ae633a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:02.471138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.472639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:02.486264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e316b1-7264-4b42-bd99-1a537fbc36fe admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.545001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.545863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.546141 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.546413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] PATH_INFO: `/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.547609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bac0e2f07b444a2826a42e8ca1942dc)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.549613 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.597506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bac0e2f07b444a2826a42e8ca1942dc', 'request_id': 'req-7c06193d-7ef5-4593-885f-25a9ebd8edbf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.603567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bac0e2f07b444a2826a42e8ca1942dc', 'request_id': 'req-7c06193d-7ef5-4593-885f-25a9ebd8edbf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.612696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bac0e2f07b444a2826a42e8ca1942dc', 'request_id': 'req-7c06193d-7ef5-4593-885f-25a9ebd8edbf'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.622867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] The token cache is being invalidate because project 1bac0e2f07b444a2826a42e8ca1942dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:02.623214 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.623953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7c06193d-7ef5-4593-885f-25a9ebd8edbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:02.625544 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1292/2619] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:02 2022] DELETE /identity/v3/projects/1bac0e2f07b444a2826a42e8ca1942dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:02.630947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.631426 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.631502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.650967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:02.652208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:02.718970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1328/2620] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:02.726645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.726946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.727244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.746300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:02.747507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:02.756105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1293/2621] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:02.760545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.770557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ee3f-f880-4411-8d15-ea14e704a1de None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.775445 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.776098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.776410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.776688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.777413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:02.777695 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.779322 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.791487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1329/2622] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:29:02.795738 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.805431 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4526-a9bb-4e05-8455-cad403f6048b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.811243 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.811994 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.812454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.812831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.813508 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.815751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.846059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1294/2623] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:02 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:02.850497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.861785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83cbef-a7c5-44a6-9105-3903caa89262 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.927935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.928546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.928828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.929126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.929851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.931754 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.954354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1330/2624] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:02 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:02.958837 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:02.969651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd7-f365-46a9-a59e-935a2a3a18a3 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:02.975754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:02.976517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:02.976820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:02.977112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:02.977975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:02.979699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:02.994554 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1295/2625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:02.998750 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.010590 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abd155-2d00-4aa0-bad4-397bfdbbdaec admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.016400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.017069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.017359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.017636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.018960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.021054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.032283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1331/2626] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:03.037172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.048083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aade1e-a0ab-4247-9aa2-2c558ad3a9f1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.054424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.056127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.056266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.057170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.057512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.057862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] PATH_INFO: `/v3/projects/a93ff1901ada4f9e873c9c03ef8923dd/users/de67b79683664c33a84c64e2f87922fa/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.069860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8646-23db-4da6-ad52-b426a2ec9dfc admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.073609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93ff1901ada4f9e873c9c03ef8923dd, user_id=de67b79683664c33a84c64e2f87922fa, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.075755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.076373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93050ee-e3ed-412e-b822-abcbb7a499f8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.076779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.077064 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.077340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] PATH_INFO: `/v3/users/294d1c4723964b3cb145a74c3c68f91d` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.078555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=294d1c4723964b3cb145a74c3c68f91d)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.080619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.101673 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1296/2627] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:03 2022] PUT /identity/v3/projects/a93ff1901ada4f9e873c9c03ef8923dd/users/de67b79683664c33a84c64e2f87922fa/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:03.103481 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.103867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.104382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.132954 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] MFA Rules not processed for user `de67b79683664c33a84c64e2f87922fa`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:03.134164 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.138966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294d1c4723964b3cb145a74c3c68f91d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.152220 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294d1c4723964b3cb145a74c3c68f91d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.153075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294d1c4723964b3cb145a74c3c68f91d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.153770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294d1c4723964b3cb145a74c3c68f91d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.161305 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-3804346d-5bb2-4983-b280-2be4646c7d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294d1c4723964b3cb145a74c3c68f91d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.175743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1332/2628] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:03 2022] DELETE /identity/v3/users/294d1c4723964b3cb145a74c3c68f91d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:03.180235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.180675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.181002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.197845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:03.199030 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.201059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1297/2629] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:03.205610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.207111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.262803 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1333/2630] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:03.272817 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75117c8-28fc-47c6-8d64-2db6dcd730a0 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.273900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.275678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.277538 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.278212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.278504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.279455 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.280183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.282127 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.311307 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1298/2631] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:03.315513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.338655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dbd1c-3aa9-47d6-8a78-ef00330b15cb None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.382946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f7e85-bc3b-49b4-96d3-1ecd37049674 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.387656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.388360 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.388646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.388919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.389600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.389954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.390663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.391006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.391343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.391531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.400586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.402086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.411314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1299/2632] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:03.414839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1334/2633] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:03.415228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.425262 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cafa-687a-42ce-9b0a-8e3b42cc826e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.430645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.431290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.431568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.431843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.432600 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.434247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.445499 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1300/2634] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:03.450676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.461187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1341073-3e8c-42a1-bcb8-493d41941c31 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.466083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.466976 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.467276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.467559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] PATH_INFO: `/v3/projects/bb2f8e78c120493b8da7440d5084a180/users/e1d9d247b1dc420ca6e72dea1cf98075/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.484723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2f8e78c120493b8da7440d5084a180, user_id=e1d9d247b1dc420ca6e72dea1cf98075, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.488517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.505852 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1335/2635] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:03 2022] PUT /identity/v3/projects/bb2f8e78c120493b8da7440d5084a180/users/e1d9d247b1dc420ca6e72dea1cf98075/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:03.509851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.521051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925c4f-d9fe-471c-804d-bf9b9fe084f5 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.578794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.579477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.579757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.580031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.581618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.583259 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a688a76-1991-425e-b63d-80cee7f9175e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.593242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1301/2636] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:03.598480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.608424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ca2d-93ca-43ff-a827-c2c314231181 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.613987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.614872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.615150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.615421 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] PATH_INFO: `/v3/projects/bb2f8e78c120493b8da7440d5084a180/users/e1d9d247b1dc420ca6e72dea1cf98075/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.617448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2f8e78c120493b8da7440d5084a180, user_id=e1d9d247b1dc420ca6e72dea1cf98075, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.620243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.631296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1336/2637] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:03 2022] PUT /identity/v3/projects/bb2f8e78c120493b8da7440d5084a180/users/e1d9d247b1dc420ca6e72dea1cf98075/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:03.642566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.642901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.643183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.671843 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] MFA Rules not processed for user `e1d9d247b1dc420ca6e72dea1cf98075`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:03.673147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.687466 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.689377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.749616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1302/2638] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:03.760391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.761226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c02d9-0925-4db3-a68a-63e7f800638c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.761980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.771820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.772766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.773048 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.773321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] PATH_INFO: `/v3/projects/3c43d22712ef48ada5bb2d940efc3372` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.774510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c43d22712ef48ada5bb2d940efc3372)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.776438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.826070 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2be3c0-2c92-4115-a5d3-34d33ac77fe3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:03.826477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c43d22712ef48ada5bb2d940efc3372', 'request_id': 'req-70e796b7-53f5-4663-b850-16242b2fba3a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.832949 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c43d22712ef48ada5bb2d940efc3372', 'request_id': 'req-70e796b7-53f5-4663-b850-16242b2fba3a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.841510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c43d22712ef48ada5bb2d940efc3372', 'request_id': 'req-70e796b7-53f5-4663-b850-16242b2fba3a'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.853243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] The token cache is being invalidate because project 3c43d22712ef48ada5bb2d940efc3372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:03.853533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.854211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-70e796b7-53f5-4663-b850-16242b2fba3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:03.855797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1337/2639] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:03 2022] DELETE /identity/v3/projects/3c43d22712ef48ada5bb2d940efc3372 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:03.861085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cb37e-8d54-4fd6-98f2-43f91f0f9531 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.861541 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cb37e-8d54-4fd6-98f2-43f91f0f9531 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.862024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cb37e-8d54-4fd6-98f2-43f91f0f9531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.881502 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-331cb37e-8d54-4fd6-98f2-43f91f0f9531 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:03.882761 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-331cb37e-8d54-4fd6-98f2-43f91f0f9531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.886557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:03.887225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.887513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.887790 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.889597 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.942538 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1338/2640] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:03.950971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:03.951288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:03.951609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:03.953509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:03.954955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:03.964614 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1303/2641] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:03.969828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:03.971200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:03.980298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1339/2642] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:03.984773 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:03.995895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270149-db01-4f18-a240-09146b54d8b8 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.001378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.002036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.002379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.002658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.003478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:04.003797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.005433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.021117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1304/2643] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:04.023588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.034689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e756311-2e9e-4a7e-9748-68b3a1f4357a None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.039691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.040388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.040671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.040952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.041636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.043679 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.068372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.070350 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:04.073731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1340/2644] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:04.078315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.085265 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2b36-abb1-4f08-a430-da700211fdbe admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.140661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.141312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.141573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.142031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.148314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabf438-880a-4327-8c43-a63c85e3941f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.151117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:04.151550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.152387 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.152780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.153177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.153922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.155956 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.178837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1341/2645] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:04.182965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.195690 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af555f83-9083-48c7-8bd0-df8bd79c9301 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.200747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.201415 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.201693 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.201988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.202747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.204366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.214596 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1342/2646] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:04.217584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.218680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.219197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.229058 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1305/2647] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:04.229805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d83837-3f3f-4473-b7ec-cfe6b1dd0aab admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.235252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.235905 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.236184 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.236681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.237497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.239076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe5fd-b4f5-461d-97b5-a3cd1ee6b6b9 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.250853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1343/2648] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:04.255560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.266589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464f847-db1d-4901-906a-54f857d88a09 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.271217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.272109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.272438 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.272692 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] PATH_INFO: `/v3/projects/53a46fc85298494a9dd3988aa5f691fa/users/01c0191a2c5d42ebb09637fe91ba1ed8/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.287423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a46fc85298494a9dd3988aa5f691fa, user_id=01c0191a2c5d42ebb09637fe91ba1ed8, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.290264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.307094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1306/2649] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:04 2022] PUT /identity/v3/projects/53a46fc85298494a9dd3988aa5f691fa/users/01c0191a2c5d42ebb09637fe91ba1ed8/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:04.319310 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.320297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.320543 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.353027 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] MFA Rules not processed for user `01c0191a2c5d42ebb09637fe91ba1ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:04.354229 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:04.418952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1344/2650] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:04.423910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.509589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6847b7f-3748-44fd-8972-d3e994b96923 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.516321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.517488 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.517999 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.518406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.519410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.521963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.561433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1307/2651] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:04.565593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.577324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0ca61-b1fd-44ba-8ffe-e7058e3c5344 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.635408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.636116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.636369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.636646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.637338 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.639290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.660951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1345/2652] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:04.665207 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.676642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c231b-ef85-4a97-8347-9ab0cfdfe381 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.682985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.683648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.683928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.684200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.685002 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.686608 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.698224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1308/2653] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:04.702548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.713214 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bc790-8653-4289-8567-dd4c40f46d68 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.718997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.719902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.720182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.720490 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] PATH_INFO: `/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7/users/14bc709a96fe41ce9fd071e7f9153de9/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.739147 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfabe62f0d24f659a3a3fb05044b9c7, user_id=14bc709a96fe41ce9fd071e7f9153de9, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.743234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.759291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1346/2654] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:04 2022] PUT /identity/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7/users/14bc709a96fe41ce9fd071e7f9153de9/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:04.763483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.773924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42f96-ab0f-491c-b053-02f8a59fb5ed admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.832696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.833368 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.833647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.833952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.834681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.836298 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88518815-bde1-45d1-94d9-ef18c058b9a4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.846581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1309/2655] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:04.850989 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:04.861756 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aa005-83e8-4768-8842-7164e10fc021 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:04.867692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:04.868677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.868961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.869233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] PATH_INFO: `/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7/users/14bc709a96fe41ce9fd071e7f9153de9/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.871460 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfabe62f0d24f659a3a3fb05044b9c7, user_id=14bc709a96fe41ce9fd071e7f9153de9, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:04.874213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:04.885585 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1347/2656] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:04 2022] PUT /identity/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7/users/14bc709a96fe41ce9fd071e7f9153de9/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:04.896324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:04.896657 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:04.897044 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:04.935739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] MFA Rules not processed for user `14bc709a96fe41ce9fd071e7f9153de9`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:04.937444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:05.008435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1310/2657] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:05.019990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:05.021941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:05.080242 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcc1e2-bda3-4205-b793-eec2bf0576ce admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:05.084446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:05.085084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:05.085361 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:05.085645 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:05.094805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:05.096584 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:05.105783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1348/2658] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:05.188545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:05.199566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8b749-00bb-488c-a2a9-9c2ec3e9cf01 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:05.204387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:05.205121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:05.205467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:05.205825 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:05.265099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:05.266502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28b84d-7733-4c04-a9de-82785d0ceb1f service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:05.275493 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1311/2659] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:08.952139 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:08.953743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:09.022662 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c181a-79a0-49ee-9eab-dfe114872830 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:09.027859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:09.028836 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:09.029185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:09.029461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] PATH_INFO: `/v3/users/de67b79683664c33a84c64e2f87922fa` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:09.030731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de67b79683664c33a84c64e2f87922fa)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:09.032718 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:09.086151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de67b79683664c33a84c64e2f87922fa'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.103939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de67b79683664c33a84c64e2f87922fa'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.104950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de67b79683664c33a84c64e2f87922fa'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.105843 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de67b79683664c33a84c64e2f87922fa'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.114874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de67b79683664c33a84c64e2f87922fa'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.135001 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1349/2660] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:08 2022] DELETE /identity/v3/users/de67b79683664c33a84c64e2f87922fa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:09.136852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:09.137317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:09.137490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:09.155574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:09.156804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:09.216671 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1312/2661] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:09.226710 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:09.228211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:09.287758 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2825e31-ff3b-4c7a-a72d-6801f2ac45e5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:09.291912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:09.292606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:09.292888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:09.293165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:09.302951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:09.304588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:09.314209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1350/2662] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:09.593530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:09.595215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:09.614609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8e397-a950-4dc4-85dc-11ce354f2972 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:09.621361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:09.622566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:09.622929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:09.623297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] PATH_INFO: `/v3/projects/a93ff1901ada4f9e873c9c03ef8923dd` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:09.624604 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93ff1901ada4f9e873c9c03ef8923dd)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:09.626595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:09.674248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93ff1901ada4f9e873c9c03ef8923dd', 'request_id': 'req-268b8a94-bbc6-4e68-a7c1-2ae65e361606'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.680858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93ff1901ada4f9e873c9c03ef8923dd', 'request_id': 'req-268b8a94-bbc6-4e68-a7c1-2ae65e361606'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.689546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93ff1901ada4f9e873c9c03ef8923dd', 'request_id': 'req-268b8a94-bbc6-4e68-a7c1-2ae65e361606'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.700403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] The token cache is being invalidate because project a93ff1901ada4f9e873c9c03ef8923dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:09.700685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.701398 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.703010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1313/2663] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:09 2022] DELETE /identity/v3/projects/a93ff1901ada4f9e873c9c03ef8923dd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:09.707201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:09.716936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b832-0a72-4b8a-8230-935bff82675f service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:09.775287 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:09.776186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:09.776500 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:09.776772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] PATH_INFO: `/v3/users/e1d9d247b1dc420ca6e72dea1cf98075` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:09.778065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d9d247b1dc420ca6e72dea1cf98075)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:09.780054 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:09.835648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d9d247b1dc420ca6e72dea1cf98075'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.849595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d9d247b1dc420ca6e72dea1cf98075'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.854859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d9d247b1dc420ca6e72dea1cf98075'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.854859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d9d247b1dc420ca6e72dea1cf98075'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.861868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-39863c9e-7a01-4cb8-b294-dc1d6760fd0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d9d247b1dc420ca6e72dea1cf98075'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:09.881953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1351/2664] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:09 2022] DELETE /identity/v3/users/e1d9d247b1dc420ca6e72dea1cf98075 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:10.159063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.160593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:10.174766 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8a94-bbc6-4e68-a7c1-2ae65e361606 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.237721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.238549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.238851 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.239102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] PATH_INFO: `/v3/projects/bb2f8e78c120493b8da7440d5084a180` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.240310 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2f8e78c120493b8da7440d5084a180)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.242346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.289435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2f8e78c120493b8da7440d5084a180', 'request_id': 'req-215d65ef-5819-434f-88cd-8e906a5a9755'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:10.295545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2f8e78c120493b8da7440d5084a180', 'request_id': 'req-215d65ef-5819-434f-88cd-8e906a5a9755'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:10.304891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2f8e78c120493b8da7440d5084a180', 'request_id': 'req-215d65ef-5819-434f-88cd-8e906a5a9755'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:10.315293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] The token cache is being invalidate because project bb2f8e78c120493b8da7440d5084a180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:10.315570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:10.316221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-215d65ef-5819-434f-88cd-8e906a5a9755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:10.317801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1314/2665] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:10 2022] DELETE /identity/v3/projects/bb2f8e78c120493b8da7440d5084a180 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:10.325727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.326473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.326611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.348603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:10.349881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:10.417177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1352/2666] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:10.425564 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.425880 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.426159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.446426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:10.447648 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:10.457248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1315/2667] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:10.461745 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.471992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42476651-640a-4c73-8d20-f170ef17fefd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.479188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.479859 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.480148 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.480457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.481227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:10.481498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.483393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.496120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1353/2668] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:10.500667 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.511772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efed64-b648-49a0-bcd9-5742afcd03d2 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.517176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.517816 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.518094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.518365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.519045 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.520988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.554320 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1316/2669] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:10.559588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.573935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bffb-8ebd-4edf-bf4f-ec3e0b8d34d7 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.634278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.634977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.635253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.635678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.636288 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.638177 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.658651 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1354/2670] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/users => generated 305 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:10.662580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.672775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d66941-0280-45b7-9d21-8194d5068995 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.677271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.677928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.678208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.678483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.679203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.680956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.691150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1317/2671] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:10.695276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.706554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e5551-135b-4f85-8640-1198b4378856 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.712892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.713595 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.713885 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.714146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.714880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.716507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eee4cb-bee0-4971-831e-c13963ab28db admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.727377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1355/2672] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:10.731969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.745409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031bbe-30d1-40dc-b0c2-7e2e35e39ba9 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.751075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.751894 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.752171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.752475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] PATH_INFO: `/v3/projects/a213ebaf36ff41fba59db1acbe1c3247/users/c5c96c9ca2c0468cb1ec6a9dd3982433/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.770237 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a213ebaf36ff41fba59db1acbe1c3247, user_id=c5c96c9ca2c0468cb1ec6a9dd3982433, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:10.772985 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:10.794102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1318/2673] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:10 2022] PUT /identity/v3/projects/a213ebaf36ff41fba59db1acbe1c3247/users/c5c96c9ca2c0468cb1ec6a9dd3982433/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:10.808551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.808846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:10.809116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:10.841696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] MFA Rules not processed for user `c5c96c9ca2c0468cb1ec6a9dd3982433`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:10.843440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:10.921368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1356/2674] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:10.927675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:10.993286 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e4b09-4582-4d4b-bf64-ba05a2ace029 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:10.998937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:10.999602 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:10.999890 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.000169 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.000884 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.002870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.030098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1319/2675] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:10 2022] POST /identity/v3/projects => generated 407 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:11.034432 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.045568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe8728-4130-44f8-bbf7-76665a96d153 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.102112 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.102789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.103081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.103354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.104033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.105962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.127643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1357/2676] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:11 2022] POST /identity/v3/users => generated 310 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:11.131875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.143486 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90261e2b-b029-4c89-8305-36283a018d24 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.155517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.156236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.156547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.156824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.157571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.159203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.170198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1320/2677] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:11.174524 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.184540 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31437712-c58c-4799-aa45-8f6fcad4f524 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.189188 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.190099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.190377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.190648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] PATH_INFO: `/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d/users/f3027062a69f4130bab3ed2b58fedc0b/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.206416 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbab63f1e4748ddad7a3aef33a78c9d, user_id=f3027062a69f4130bab3ed2b58fedc0b, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.209151 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.224986 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1358/2678] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:11 2022] PUT /identity/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d/users/f3027062a69f4130bab3ed2b58fedc0b/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:11.231103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.242129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b3269-817b-43d3-bc58-02340c248bf2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.300588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.301276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.301559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.301839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.302570 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.304187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a7e27-0950-4883-922d-75fa9ea37ee4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.314441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1321/2679] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:11.321495 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.332319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5755-174b-4d5a-9189-0348840f2e01 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.337603 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.338492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.338776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.339042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] PATH_INFO: `/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d/users/f3027062a69f4130bab3ed2b58fedc0b/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.341085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbab63f1e4748ddad7a3aef33a78c9d, user_id=f3027062a69f4130bab3ed2b58fedc0b, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.343747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.354701 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1359/2680] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:11 2022] PUT /identity/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d/users/f3027062a69f4130bab3ed2b58fedc0b/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:11.366238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.366531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.366819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.393606 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] MFA Rules not processed for user `f3027062a69f4130bab3ed2b58fedc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:11.394819 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:11.469006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1322/2681] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:11.480379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.482357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:11.541393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db795-1849-42a2-89a5-1af700a995a3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.545966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.546570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.546848 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.547244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.557316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.558689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.569043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1360/2682] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:11.745949 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:11.757300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30b10d-1670-4af1-a8e1-6c57f0f557ed None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:11.762083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:11.762719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:11.762984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:11.763275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:11.824906 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:11.826285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfea4c2-c067-4e0c-913c-ca8d61336acf service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:11.835718 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1323/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:31.897743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:31.899355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:31.967414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7074c-54f6-4360-b25b-c6d2daa3dcf6 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:31.972962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:31.973860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:31.974132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:31.974397 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] PATH_INFO: `/v3/users/01c0191a2c5d42ebb09637fe91ba1ed8` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:31.975586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c0191a2c5d42ebb09637fe91ba1ed8)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:31.977536 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:32.027055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c0191a2c5d42ebb09637fe91ba1ed8'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.038766 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c0191a2c5d42ebb09637fe91ba1ed8'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.039481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c0191a2c5d42ebb09637fe91ba1ed8'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.040140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c0191a2c5d42ebb09637fe91ba1ed8'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.047278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c0191a2c5d42ebb09637fe91ba1ed8'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.062893 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1361/2684] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:31 2022] DELETE /identity/v3/users/01c0191a2c5d42ebb09637fe91ba1ed8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:32.067588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:32.068097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:32.068187 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:32.089775 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:32.091176 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:32.156679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1324/2685] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:32.164944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:32.166389 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:32.251539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941db244-a098-4836-88bf-806d97deb97c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:32.255330 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:32.255978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:32.256259 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:32.256559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:32.265862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:32.267276 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:32.276660 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1362/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:32.552643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:32.554383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:32.567594 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c6ab6-c213-4cac-af70-0dfe6364d85b None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:32.574804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:32.575719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:32.576018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:32.576309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] PATH_INFO: `/v3/projects/53a46fc85298494a9dd3988aa5f691fa` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:32.577747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a46fc85298494a9dd3988aa5f691fa)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:32.580093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:32.634532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a46fc85298494a9dd3988aa5f691fa', 'request_id': 'req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.640450 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a46fc85298494a9dd3988aa5f691fa', 'request_id': 'req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.649429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a46fc85298494a9dd3988aa5f691fa', 'request_id': 'req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.661383 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] The token cache is being invalidate because project 53a46fc85298494a9dd3988aa5f691fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:32.661672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.662318 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.663908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1325/2687] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:32 2022] DELETE /identity/v3/projects/53a46fc85298494a9dd3988aa5f691fa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:32.669307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:32.679311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fdc5-d465-4bef-a36d-85ae3da088d9 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:32.737133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:32.738029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:32.738368 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:32.738654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] PATH_INFO: `/v3/users/14bc709a96fe41ce9fd071e7f9153de9` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:32.740352 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bc709a96fe41ce9fd071e7f9153de9)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:32.742201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:32.793696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bc709a96fe41ce9fd071e7f9153de9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.805681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bc709a96fe41ce9fd071e7f9153de9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.806326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bc709a96fe41ce9fd071e7f9153de9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.806928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bc709a96fe41ce9fd071e7f9153de9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.813942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-844fa647-5eeb-4e4c-925c-bc10b6ba7702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bc709a96fe41ce9fd071e7f9153de9'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:32.830244 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1363/2688] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:32 2022] DELETE /identity/v3/users/14bc709a96fe41ce9fd071e7f9153de9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:32.863664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:32.865251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:32.878054 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152dbb-2b6f-45c8-b30b-9e1c3d2d992a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:32.934943 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:32.935856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:32.936131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:32.936434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] PATH_INFO: `/v3/users/c5c96c9ca2c0468cb1ec6a9dd3982433` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:32.937624 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c96c9ca2c0468cb1ec6a9dd3982433)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:32.939552 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:32.989125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c96c9ca2c0468cb1ec6a9dd3982433'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.002573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c96c9ca2c0468cb1ec6a9dd3982433'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.009151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c96c9ca2c0468cb1ec6a9dd3982433'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.009893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c96c9ca2c0468cb1ec6a9dd3982433'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.018146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c96c9ca2c0468cb1ec6a9dd3982433'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.024194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1326/2689] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:32 2022] DELETE /identity/v3/users/c5c96c9ca2c0468cb1ec6a9dd3982433 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:33.029010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780ff3-b440-417c-866c-91b96425472e None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.029505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780ff3-b440-417c-866c-91b96425472e None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.029612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780ff3-b440-417c-866c-91b96425472e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.046725 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c8780ff3-b440-417c-866c-91b96425472e None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:33.047932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c8780ff3-b440-417c-866c-91b96425472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.100586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.102216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.124837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1364/2690] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:33.134667 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8780ff3-b440-417c-866c-91b96425472e None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.136506 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c8780ff3-b440-417c-866c-91b96425472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.180887 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7156-33dd-48db-9ac2-f025cc8ad847 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.187677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.188580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.189088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.189217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] PATH_INFO: `/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.190361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfabe62f0d24f659a3a3fb05044b9c7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.192281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.202033 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8780ff3-b440-417c-866c-91b96425472e None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.206850 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.207573 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.207861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.208135 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.218568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.220439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c04ca-361c-455e-bbd4-f46550273c86 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.232181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1365/2691] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:33.259861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfabe62f0d24f659a3a3fb05044b9c7', 'request_id': 'req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.269384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfabe62f0d24f659a3a3fb05044b9c7', 'request_id': 'req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.276530 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfabe62f0d24f659a3a3fb05044b9c7', 'request_id': 'req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.287978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] The token cache is being invalidate because project 7dfabe62f0d24f659a3a3fb05044b9c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:33.288359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.290234 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-a718d94d-0ac9-4338-8fd8-f57a849dfbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.292142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1327/2692] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:33 2022] DELETE /identity/v3/projects/7dfabe62f0d24f659a3a3fb05044b9c7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:33.297126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.297451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.297742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.320565 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:33.321767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.392404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1366/2693] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:33.399540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.399824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.400098 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.418668 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:33.419888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.431316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1328/2694] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:33.432542 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.442257 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f7a7f-d916-4c9e-b370-232f5e11e07b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.447358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.448009 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.448358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.448627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.449357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:33.449665 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.451269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.463796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1367/2695] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:33.467541 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.476986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68af2c-3fb9-4263-a23a-5c134adec1cc None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.482980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.483676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.483982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.484332 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.484952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.486911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.516172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1329/2696] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:33.517783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.522695 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.524494 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.528237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21589b-045e-4599-bd50-ac6693708bb2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.587532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c165-bbb3-4ec0-9398-d45eed409338 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.592379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.593512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.593786 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.593883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] PATH_INFO: `/v3/projects/a213ebaf36ff41fba59db1acbe1c3247` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.595071 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a213ebaf36ff41fba59db1acbe1c3247)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.595871 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.596780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.597072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.597226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.597337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.598035 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.599972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.620801 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1368/2697] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/users => generated 280 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:33.624924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.635021 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af6ec3-ab93-4292-b83c-bf14b33fc581 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.647776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a213ebaf36ff41fba59db1acbe1c3247', 'request_id': 'req-1b924367-4921-4704-bca2-d18440f757a2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.654150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a213ebaf36ff41fba59db1acbe1c3247', 'request_id': 'req-1b924367-4921-4704-bca2-d18440f757a2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.662881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a213ebaf36ff41fba59db1acbe1c3247', 'request_id': 'req-1b924367-4921-4704-bca2-d18440f757a2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.673874 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] The token cache is being invalidate because project a213ebaf36ff41fba59db1acbe1c3247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:33.674238 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.674992 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.676767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1330/2698] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:33 2022] DELETE /identity/v3/projects/a213ebaf36ff41fba59db1acbe1c3247 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:33.680568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.682179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.694018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.694694 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.694975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.695602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.696379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.697966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.708051 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1369/2699] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:33.711943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.713688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.726032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cde409-2732-4355-9967-87c4733fc910 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.730684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.731335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.731597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.731908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.732670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.734285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.747865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1370/2700] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:33.748101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b924367-4921-4704-bca2-d18440f757a2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.749533 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.751802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.752747 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.752998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.753308 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] PATH_INFO: `/v3/users/f3027062a69f4130bab3ed2b58fedc0b` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.754654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3027062a69f4130bab3ed2b58fedc0b)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.756617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.759792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95b7e3-5b6c-47ef-a666-b2e9e33f82c3 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:33.764749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:33.765640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.765929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.766211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] PATH_INFO: `/v3/projects/d2023066ff3f4f0e949e12070b542959/users/3a8bfa6be0f143efbe372d0c6883b5b6/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.782522 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2023066ff3f4f0e949e12070b542959, user_id=3a8bfa6be0f143efbe372d0c6883b5b6, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:33.785611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bfbbb-239b-4ed6-a299-b1c23a7cbf06 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:33.810386 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1371/2701] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:33 2022] PUT /identity/v3/projects/d2023066ff3f4f0e949e12070b542959/users/3a8bfa6be0f143efbe372d0c6883b5b6/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:33.816157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3027062a69f4130bab3ed2b58fedc0b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.830881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3027062a69f4130bab3ed2b58fedc0b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.838436 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3027062a69f4130bab3ed2b58fedc0b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.839220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3027062a69f4130bab3ed2b58fedc0b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.849649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3027062a69f4130bab3ed2b58fedc0b'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:33.856174 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1331/2702] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:33 2022] DELETE /identity/v3/users/f3027062a69f4130bab3ed2b58fedc0b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:33.866118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:33.866410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:33.866692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:33.895089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] MFA Rules not processed for user `3a8bfa6be0f143efbe372d0c6883b5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:33.896474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:33.961324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1372/2703] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:33 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:33.992993 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:33.994406 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.055447 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdb4-f05c-422b-a8c4-0d3484e247cb admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.059197 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.059849 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.060124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.060429 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.069643 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.071679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.081618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1332/2704] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:34.095411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.097137 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.128848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.130277 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.170879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efef7f-3000-44c9-9086-cd6ed585c38b None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.175391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.176291 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.176580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.176842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] PATH_INFO: `/v3/users/3a8bfa6be0f143efbe372d0c6883b5b6` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.178039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8bfa6be0f143efbe372d0c6883b5b6)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.179959 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.195539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b716e4-b184-418f-a067-ed32578da836 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.201170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.202079 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.202339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.202615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] PATH_INFO: `/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.204185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbab63f1e4748ddad7a3aef33a78c9d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.206519 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.227224 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8bfa6be0f143efbe372d0c6883b5b6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.239252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8bfa6be0f143efbe372d0c6883b5b6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.239932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8bfa6be0f143efbe372d0c6883b5b6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.240558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8bfa6be0f143efbe372d0c6883b5b6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.247071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-5567d51b-fb47-4c43-850a-36fbf96554a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8bfa6be0f143efbe372d0c6883b5b6'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.255102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbab63f1e4748ddad7a3aef33a78c9d', 'request_id': 'req-283103e0-932c-48cb-9a55-bf008ca7acb1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.261173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbab63f1e4748ddad7a3aef33a78c9d', 'request_id': 'req-283103e0-932c-48cb-9a55-bf008ca7acb1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.261320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1373/2705] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:34 2022] DELETE /identity/v3/users/3a8bfa6be0f143efbe372d0c6883b5b6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:34.265741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.266056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.266362 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.269170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbab63f1e4748ddad7a3aef33a78c9d', 'request_id': 'req-283103e0-932c-48cb-9a55-bf008ca7acb1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.284179 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.285049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] The token cache is being invalidate because project 5fbab63f1e4748ddad7a3aef33a78c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:34.285375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.285519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.286149 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-283103e0-932c-48cb-9a55-bf008ca7acb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.287805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1333/2706] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:34 2022] DELETE /identity/v3/projects/5fbab63f1e4748ddad7a3aef33a78c9d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:34.293011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d72-7983-4587-af49-9a90e67cb8d8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.293297 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d72-7983-4587-af49-9a90e67cb8d8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.293574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d72-7983-4587-af49-9a90e67cb8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.313075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-491c0d72-7983-4587-af49-9a90e67cb8d8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.314288 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-491c0d72-7983-4587-af49-9a90e67cb8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.354325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1374/2707] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:34.365211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.366597 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.372337 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1334/2708] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:34.380101 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.380531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.380699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.399616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.400847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.409513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1335/2709] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:34.413335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.422143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f933-7122-47fe-870d-c6817d2abe3a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.426932 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b396121-8883-49ca-b54f-e04fca56fb60 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.427370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.428028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.428353 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.428627 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.429367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:34.429651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.430771 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.431247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.431406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.431681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.431951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.440944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.442325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.443402 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1336/2710] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:29:34.447829 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.452038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1375/2711] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:34.457814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f9a22-6a51-4500-afe3-2a522aa797c8 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.462727 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.463376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.463658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.463933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.464656 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.466562 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.494441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1337/2712] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:34.498512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.509118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820a04d-dde7-4d0a-9fbb-43e5b92608df service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.565851 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.566684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.567061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.567464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.568283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.570612 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.593350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1376/2713] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:34.598091 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.608848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6e36-35a3-4098-bd3e-6b4e9af1774f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.613805 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.614545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.614888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.615226 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.616018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.617720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.627820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1338/2714] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:34.632044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.642270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ffee3-08b9-40eb-abfd-46ca25a3ee19 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.647366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.648043 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.648356 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.648640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.649379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.651120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.661993 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1377/2715] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:34.666033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.676381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7147c99-f69c-4638-b5df-a572f6d55297 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.681393 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.682312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.682590 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.682895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] PATH_INFO: `/v3/projects/20f14b7d2c3a4fd78ecc16256f3aab75/users/df7cfb1f18484852bb066354d5395dac/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.698018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f14b7d2c3a4fd78ecc16256f3aab75, user_id=df7cfb1f18484852bb066354d5395dac, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.700827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b99f8-1207-4333-bb4f-1c1f17bd2af5 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.717435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1339/2716] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:34 2022] PUT /identity/v3/projects/20f14b7d2c3a4fd78ecc16256f3aab75/users/df7cfb1f18484852bb066354d5395dac/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:34.718792 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.720359 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.720637 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.720764 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.721203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.733967 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c6171-b6f0-4506-b645-1144c1ca6f71 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.740918 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.742417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.799199 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.800087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.800436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.800715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] PATH_INFO: `/v3/projects/d2023066ff3f4f0e949e12070b542959` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.801687 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1340/2717] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:34.801977 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2023066ff3f4f0e949e12070b542959)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.803925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.811264 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:34.812635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.852804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2023066ff3f4f0e949e12070b542959', 'request_id': 'req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.859340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2023066ff3f4f0e949e12070b542959', 'request_id': 'req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.868191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2023066ff3f4f0e949e12070b542959', 'request_id': 'req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.878752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] The token cache is being invalidate because project d2023066ff3f4f0e949e12070b542959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:34.878972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.879633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6ba5353d-7080-44f6-ada6-cdbee90fcf4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:34.881062 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904011a-fe44-4f61-b137-ae186ff359f3 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:34.881323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1378/2718] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:34 2022] DELETE /identity/v3/projects/d2023066ff3f4f0e949e12070b542959 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:34.886309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664515d7-799b-43ac-8b57-43d02249bb3f None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.886705 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664515d7-799b-43ac-8b57-43d02249bb3f None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.887057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664515d7-799b-43ac-8b57-43d02249bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.905075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-664515d7-799b-43ac-8b57-43d02249bb3f None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.906743 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-664515d7-799b-43ac-8b57-43d02249bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.929440 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:34.930129 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.930410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.930682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.932719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.964190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1379/2719] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:34.971652 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:34.971942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:34.972221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:34.989909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:34.991126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:34.994726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:34.996312 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:34.999978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1380/2720] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:35.004324 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.006100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1341/2721] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:35.013357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e74e1-6680-4ec1-b6ca-f977437df0e7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.018354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.019015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.019299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.019575 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.020379 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:35.020681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.022274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.034579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1381/2722] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:35.038855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.048511 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4f421-65a5-47d9-ab38-24531f7167f8 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.053392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.054147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.054430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.054708 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.055446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.057411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.089189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1342/2723] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:35.093644 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.104402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da585b3a-51e2-4f1f-8120-3710bc7585ac admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.161342 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.162025 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.162309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.162582 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.163269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.165209 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.190777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1382/2724] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:35.192059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.203645 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a38ae-592a-4063-832f-fa2838190b8c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.208961 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.209630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.209909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.210179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.210910 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.212566 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.224839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1343/2725] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:35.228982 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.239393 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0f846-d0f9-495c-9b96-44cd5204bc03 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.244539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.245348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.245630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.245901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.246621 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.248240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8580c8-2138-434e-a2a8-130bef003e3a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.258654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1383/2726] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:35.262784 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.274411 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec092d-6960-4baf-b5a8-b0e4118d7895 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.279376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.281680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.281680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.281680 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] PATH_INFO: `/v3/projects/88b85c0285b74fb69e7fc861e42227e9/users/054cd63ee3e643a0a3ed7ca0f58d5bd2/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.297401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b85c0285b74fb69e7fc861e42227e9, user_id=054cd63ee3e643a0a3ed7ca0f58d5bd2, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.300113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.313632 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1344/2727] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:35 2022] PUT /identity/v3/projects/88b85c0285b74fb69e7fc861e42227e9/users/054cd63ee3e643a0a3ed7ca0f58d5bd2/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:35.326428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.326720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.327003 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.383545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] MFA Rules not processed for user `054cd63ee3e643a0a3ed7ca0f58d5bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:35.385406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:35.461996 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1384/2728] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:35.466679 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.528013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e7496-552f-4cd4-b308-f4a97b2627ba admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.532742 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.533412 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.533694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.534131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.534844 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.536800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27640a0-1558-486a-bebf-5a1a4663243b admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.563904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1345/2729] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:35.568508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.579643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b375b-d9fb-41da-a062-d7896b5d0e1c None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.649245 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.649935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.650250 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.650525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.651223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.652988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.653388 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.653768 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.653873 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.679251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1385/2730] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:35.683391 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.684732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] MFA Rules not processed for user `df7cfb1f18484852bb066354d5395dac`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:35.686041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:35.693937 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0084e8-408f-4551-8ede-d6132d162175 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.699109 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.699853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.700192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.700558 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.701360 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.703002 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.716348 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1386/2731] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:35.719909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.729909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136146f-1ab1-4f47-a149-22273aa323aa admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.734433 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.735098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.735377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.735649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.736411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.738084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.749058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1387/2732] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:35.753732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.762916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1346/2733] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:35.764320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867450c4-4972-4532-a446-663d329d5129 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.769703 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.770709 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.771057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.771405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] PATH_INFO: `/v3/projects/6010e68df762421696f450cd85e89d78/users/ffb85104246d411aba2069f5aa5f5efb/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.774557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.776900 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:35.788385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6010e68df762421696f450cd85e89d78, user_id=ffb85104246d411aba2069f5aa5f5efb, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.791162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6fd89-f43a-46be-9a27-a37f8d3b7153 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.816888 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1388/2734] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:35 2022] PUT /identity/v3/projects/6010e68df762421696f450cd85e89d78/users/ffb85104246d411aba2069f5aa5f5efb/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:35.818707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.819072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.819557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.840221 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7552761-e43d-408c-b79f-e2c0423730c4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:35.849551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] MFA Rules not processed for user `ffb85104246d411aba2069f5aa5f5efb`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:35.850841 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:35.904454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:35.905102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:35.905381 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:35.905709 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:35.940016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1389/2735] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:35.944954 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:35.967342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:35.968917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:35.978940 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1347/2736] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:36.008673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ed4ac-78f1-424a-9b56-564c0f04cd66 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.013499 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.014190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.014473 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.014752 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.015444 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.017404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.045373 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1390/2737] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:35 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:36.046896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.057537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a606d-822f-4542-83da-6d022c3507dc service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.115527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.116210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.116524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.116804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.117517 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.119848 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.144346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1348/2738] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:36 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:36.145733 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.156381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088b886-97c1-4f6b-bee1-67121a7b1b82 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.161251 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.161914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.162200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.162477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.163207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.164860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.176171 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1391/2739] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:36.180767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.193858 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c566acf-0a09-43ca-a767-c93c652c222d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.199314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.201050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.201339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.201666 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] PATH_INFO: `/v3/projects/47ecc23a20cd43a081a6a29aa2599278/users/5a351f6f84fe4f4fa979784761373e16/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.218247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ecc23a20cd43a081a6a29aa2599278, user_id=5a351f6f84fe4f4fa979784761373e16, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.221728 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.239075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1349/2740] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:36 2022] PUT /identity/v3/projects/47ecc23a20cd43a081a6a29aa2599278/users/5a351f6f84fe4f4fa979784761373e16/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:36.242849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.253203 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d39c6-65ee-4c5d-8db9-985c686b4ed2 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.320482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.321174 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.321458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.321732 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.322484 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.324534 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256726d4-8280-43bf-9c59-fc422738d914 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.335066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1392/2741] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:36.339340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.349647 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad640f-43b3-4044-9b1c-a5d655cd13a4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.354721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.355610 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.355893 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.356171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] PATH_INFO: `/v3/projects/47ecc23a20cd43a081a6a29aa2599278/users/5a351f6f84fe4f4fa979784761373e16/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.358401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ecc23a20cd43a081a6a29aa2599278, user_id=5a351f6f84fe4f4fa979784761373e16, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.361116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.374653 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1350/2742] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:36 2022] PUT /identity/v3/projects/47ecc23a20cd43a081a6a29aa2599278/users/5a351f6f84fe4f4fa979784761373e16/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:36.385794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca903a3-9f80-4b12-9c81-5d3bce1ef688 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.385895 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca903a3-9f80-4b12-9c81-5d3bce1ef688 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.386246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca903a3-9f80-4b12-9c81-5d3bce1ef688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.412784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6ca903a3-9f80-4b12-9c81-5d3bce1ef688 None None] MFA Rules not processed for user `5a351f6f84fe4f4fa979784761373e16`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:36.413990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6ca903a3-9f80-4b12-9c81-5d3bce1ef688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:36.483083 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1393/2743] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:36.496085 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:36.557982 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c21ce-f0e8-4da1-ab72-1d3c9176db55 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:36.561967 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:36.562581 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:36.562860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:36.563132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:36.619789 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:36.621254 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:36.630637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1351/2744] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:41.542358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:41.542899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:41.542899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:41.562343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] MFA Rules not processed for user `df7cfb1f18484852bb066354d5395dac`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:41.563535 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:41.621249 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1394/2745] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:42.372957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:42.382690 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbab73-1a5c-4424-bfc4-5826c3f2de56 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:42.387470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:42.388216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:42.388609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:42.388939 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:42.472996 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:42.475166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:42.498750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1352/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:43.074290 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:43.075958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:43.091821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5186af7-b86a-46d2-b61e-b08e55f410bf None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:43.096411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:43.097255 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:43.097692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:43.097971 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:43.108343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:43.109772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d8856-72a3-4a9a-90e9-6fa9c7290784 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:43.120066 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1395/2747] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:43.652515 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:43.654236 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:43.728177 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82fa99-820a-42ee-ab9a-fd5a5c3c25ac service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:43.733309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:43.734225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:43.734471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:43.734756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] PATH_INFO: `/v3/users/26ef4f18d9f84f59aac68b44739381fe` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:43.735979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ef4f18d9f84f59aac68b44739381fe)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:43.738050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:43.788616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ef4f18d9f84f59aac68b44739381fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:43.799716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ef4f18d9f84f59aac68b44739381fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:43.806669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ef4f18d9f84f59aac68b44739381fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:43.807376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ef4f18d9f84f59aac68b44739381fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:43.814422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ef4f18d9f84f59aac68b44739381fe'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:43.821285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1353/2748] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:43 2022] DELETE /identity/v3/users/26ef4f18d9f84f59aac68b44739381fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:43.826253 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:43.826553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:43.826819 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:43.845006 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:43.846164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:43.905847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1396/2749] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:43.917011 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:43.918588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:43.975676 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d008c-d699-419e-9c2f-d01aa2b96e9d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:43.979401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:43.980051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:43.980372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:43.980642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:43.991064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:43.992630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.004871 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1354/2750] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:44.240408 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.242968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:44.256881 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ed35a-0f8f-4f48-b4c2-acb8d9a9e8c7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.264517 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.265525 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.265675 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.266236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.268070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:44.280007 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.281720 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e75d89-45b2-4255-bf5a-cde8441f9fdc service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.290299 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1397/2751] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:44.334208 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.335700 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:44.348445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a3b6-46e4-4bce-99d0-560c1d0ee4f6 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.353203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.354082 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.354365 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.354916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] PATH_INFO: `/v3/projects/873aaccbd05a43939276cffc1b7d8b2a` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.355835 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=873aaccbd05a43939276cffc1b7d8b2a)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.357788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.420916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873aaccbd05a43939276cffc1b7d8b2a', 'request_id': 'req-bacf42ec-f884-4efb-a01f-3127eb51ab51'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:44.428060 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873aaccbd05a43939276cffc1b7d8b2a', 'request_id': 'req-bacf42ec-f884-4efb-a01f-3127eb51ab51'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:44.439454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873aaccbd05a43939276cffc1b7d8b2a', 'request_id': 'req-bacf42ec-f884-4efb-a01f-3127eb51ab51'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:44.455074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] The token cache is being invalidate because project 873aaccbd05a43939276cffc1b7d8b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:44.455582 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:44.456639 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-bacf42ec-f884-4efb-a01f-3127eb51ab51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:44.458369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1355/2752] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:44 2022] DELETE /identity/v3/projects/873aaccbd05a43939276cffc1b7d8b2a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:44.463826 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.464420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.464627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.498968 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:44.500208 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:44.589294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1398/2753] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:44.597349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.597642 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.598073 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.618484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:44.619714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:44.629477 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1356/2754] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:44.634053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.644845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373df3a-61b8-4f61-b8d1-eb976efbadea None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.649429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.650084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.650364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.650636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.651357 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:44.652037 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.653267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.664782 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1399/2755] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:29:44.669299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.679178 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e4c88-cfa9-469e-8c95-defbf88f05bb None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.684979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.685631 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.685868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.686165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.686836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.688831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.716622 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1357/2756] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:44 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:44.721385 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.732670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41a475-3fe0-4958-a8e6-01a8a26c703d admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.790163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.790487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.790773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.791046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.792013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.793697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.816226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1400/2757] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:44 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:44.821121 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.840463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534405be-1d89-4a2f-8109-4c0f94b9013f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.840463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.840463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.840463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.840743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.840743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.842016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.853006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1358/2758] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:44.857160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.867494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76b165-c7a7-44cf-9ab4-48f9ed43fdc4 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.872085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.872794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.873074 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.873346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.874121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.875707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b478a6-0370-4b29-b56a-f55fe8e1f895 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.886682 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1401/2759] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:44.890901 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:44.901290 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1d1-de0c-4a2f-885b-3400e635aa95 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:44.906425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:44.907314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.907592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.907860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] PATH_INFO: `/v3/projects/18a0553b514544e084db3af0c1523085/users/c9c4d961baad40648c787c3bbe8c3d87/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.922917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a0553b514544e084db3af0c1523085, user_id=c9c4d961baad40648c787c3bbe8c3d87, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:44.925934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:44.939652 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1359/2760] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:44 2022] PUT /identity/v3/projects/18a0553b514544e084db3af0c1523085/users/c9c4d961baad40648c787c3bbe8c3d87/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:44.951100 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac44f9c-553f-40a0-897f-a7700e857843 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:44.951470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac44f9c-553f-40a0-897f-a7700e857843 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:44.951817 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac44f9c-553f-40a0-897f-a7700e857843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:44.978212 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6ac44f9c-553f-40a0-897f-a7700e857843 None None] MFA Rules not processed for user `c9c4d961baad40648c787c3bbe8c3d87`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:44.979459 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6ac44f9c-553f-40a0-897f-a7700e857843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:45.043755 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1402/2761] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:45.056363 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:45.058213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:45.141442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01defdc0-6aa1-475a-8420-4a6399cb2c5d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:45.153615 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:45.154384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:45.154661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:45.155009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:45.167168 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:45.168836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:45.179772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1360/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:45.951505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929f19f-95f5-499c-a496-03cc7abdddfd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:45.951972 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929f19f-95f5-499c-a496-03cc7abdddfd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:45.952175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929f19f-95f5-499c-a496-03cc7abdddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:45.969076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-7929f19f-95f5-499c-a496-03cc7abdddfd None None] MFA Rules not processed for user `4b84b7192b13421286d86ba3a43d0475`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:45.970182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7929f19f-95f5-499c-a496-03cc7abdddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:46.028075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1403/2763] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:46.291828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:46.293607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:46.364891 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff273d49-b7b8-425f-9217-8555d364a2d8 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:46.369591 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:46.370501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:46.370778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:46.371080 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] PATH_INFO: `/v3/users/9442d41faa954f2dacd31c26051ffd06` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:46.372340 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9442d41faa954f2dacd31c26051ffd06)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:46.374193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:46.420772 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9442d41faa954f2dacd31c26051ffd06'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:46.432216 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9442d41faa954f2dacd31c26051ffd06'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:46.439426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9442d41faa954f2dacd31c26051ffd06'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:46.440179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9442d41faa954f2dacd31c26051ffd06'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:46.448800 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9442d41faa954f2dacd31c26051ffd06'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:46.455125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1361/2764] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:46 2022] DELETE /identity/v3/users/9442d41faa954f2dacd31c26051ffd06 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:46.460196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:46.460523 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:46.460800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:46.479514 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:46.480812 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:46.546827 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1404/2765] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:46.556678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:46.558193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:46.629203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32938982-68d7-462a-b2cf-0afbef2805f4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:46.633460 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:46.634111 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:46.634401 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:46.634660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:46.644370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:46.645841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:46.655875 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1362/2766] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:47.018833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.020375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.059246 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2844-ff2a-45db-b4dc-d44680e8b491 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.076736 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.078011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.078293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.078568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] PATH_INFO: `/v3/projects/3d2f1b69a8e94695a6db56db0e935384` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.079795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2f1b69a8e94695a6db56db0e935384)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.082878 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.091936 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.093811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.109987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c1ce-5f8b-4309-9137-4760e8272447 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.116378 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.117413 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.117595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.117868 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] PATH_INFO: `/v3/users/054cd63ee3e643a0a3ed7ca0f58d5bd2` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.119096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] RBAC: Authorizing `identity:delete_user(user_id=054cd63ee3e643a0a3ed7ca0f58d5bd2)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.121137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.162574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2f1b69a8e94695a6db56db0e935384', 'request_id': 'req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.171034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2f1b69a8e94695a6db56db0e935384', 'request_id': 'req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.183636 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2f1b69a8e94695a6db56db0e935384', 'request_id': 'req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.200427 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] The token cache is being invalidate because project 3d2f1b69a8e94695a6db56db0e935384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:47.201604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.202375 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-ed780ed4-60ac-43ed-8cd2-f5c19fc3e20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.205099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1405/2767] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:47 2022] DELETE /identity/v3/projects/3d2f1b69a8e94695a6db56db0e935384 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:47.210663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48061d9-acb7-4993-abaf-7cb3b4110c42 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.211018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48061d9-acb7-4993-abaf-7cb3b4110c42 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.211588 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48061d9-acb7-4993-abaf-7cb3b4110c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.213749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054cd63ee3e643a0a3ed7ca0f58d5bd2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.229579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054cd63ee3e643a0a3ed7ca0f58d5bd2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.232225 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-c48061d9-acb7-4993-abaf-7cb3b4110c42 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:47.233556 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c48061d9-acb7-4993-abaf-7cb3b4110c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.237001 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054cd63ee3e643a0a3ed7ca0f58d5bd2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.237826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054cd63ee3e643a0a3ed7ca0f58d5bd2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.250584 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-48f6ac76-b11f-4465-9419-4600b1e2896c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054cd63ee3e643a0a3ed7ca0f58d5bd2'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.258518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1363/2768] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:47 2022] DELETE /identity/v3/users/054cd63ee3e643a0a3ed7ca0f58d5bd2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:47.263361 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.263683 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.263929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.282677 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:47.283908 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.332240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1406/2769] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:47.342708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.343019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.343321 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.364164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:47.365404 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.366117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1364/2770] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:47.376082 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1407/2771] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:47.378731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.380502 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.380617 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.391866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02493e-b0aa-4ff6-9322-d2f9d15044e4 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.398477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.399145 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.399550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.399830 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.400601 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:47.400900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.402910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.428015 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1408/2772] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:47.429052 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.438508 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac490c6-6b0a-4d4d-b0fd-1b33be77063e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.439370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d24b-7302-4fd1-a026-630df68a1315 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.443279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.443410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.443925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.444028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.444206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.444334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.444539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.444644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.445591 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.447483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.453903 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.455373 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.465152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1365/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:47.481779 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1409/2774] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:47.486514 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.497167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa4e74-b10b-4b48-99c2-b8f44b682799 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.565978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.566685 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.566973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.567674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.568193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.570763 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.602924 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1366/2775] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:47.607842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.627202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c5b7-10d2-49a4-9c14-3343d127d4fd admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.638206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.639023 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.639312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.639939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.641232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.644580 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.663568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.663960 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1410/2776] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:47.676314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645106f-0753-4791-a72b-33113cab4d4f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.685630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.686595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.687069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.687532 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.689443 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.691601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595fdb6-1203-4273-84ab-ac2f8a3059c9 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.706370 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1367/2777] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:47.710190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.721307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399848f-e6ad-4c84-9691-e2f8a7ede44f admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.737382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.738340 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.738640 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.738904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] PATH_INFO: `/v3/projects/e8c0df7abdbe4ce7a78c0bd09da861fe/users/eaaa64eac1e14af89e2ceb7099d62025/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.760547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c0df7abdbe4ce7a78c0bd09da861fe, user_id=eaaa64eac1e14af89e2ceb7099d62025, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.765739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.790958 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1411/2778] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:47 2022] PUT /identity/v3/projects/e8c0df7abdbe4ce7a78c0bd09da861fe/users/eaaa64eac1e14af89e2ceb7099d62025/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:47.792970 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.793281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.793559 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.816341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.818039 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.826230 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] MFA Rules not processed for user `eaaa64eac1e14af89e2ceb7099d62025`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:47.827462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.881480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10de01-17cf-42a1-8b3b-c9d319faa508 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.886261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:47.887178 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:47.887461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:47.887757 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] PATH_INFO: `/v3/projects/88b85c0285b74fb69e7fc861e42227e9` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:47.888970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b85c0285b74fb69e7fc861e42227e9)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:47.890916 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:47.893248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1368/2779] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:47.903963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.905351 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:47.944317 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b85c0285b74fb69e7fc861e42227e9', 'request_id': 'req-2ec04c8f-249e-4725-9531-0f793ee8d225'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.951392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b85c0285b74fb69e7fc861e42227e9', 'request_id': 'req-2ec04c8f-249e-4725-9531-0f793ee8d225'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.960609 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b85c0285b74fb69e7fc861e42227e9', 'request_id': 'req-2ec04c8f-249e-4725-9531-0f793ee8d225'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.963895 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edc613-ea6b-41a9-a4eb-8bdfb5fc6a26 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:47.971622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] The token cache is being invalidate because project 88b85c0285b74fb69e7fc861e42227e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:47.971947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.972650 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:47.974309 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1412/2780] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:47 2022] DELETE /identity/v3/projects/88b85c0285b74fb69e7fc861e42227e9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:47.978509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:47.980262 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.015550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:48.016224 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:48.016535 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:48.016812 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:48.018739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.067769 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04c8f-249e-4725-9531-0f793ee8d225 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:48.073622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:48.074963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:48.075471 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:48.075802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] PATH_INFO: `/v3/users/ffb85104246d411aba2069f5aa5f5efb` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:48.077180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb85104246d411aba2069f5aa5f5efb)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:48.079264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:48.152997 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:48.160468 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:48.171874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb85104246d411aba2069f5aa5f5efb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.172321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1369/2781] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:48.193833 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb85104246d411aba2069f5aa5f5efb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.195865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb85104246d411aba2069f5aa5f5efb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.196832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb85104246d411aba2069f5aa5f5efb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.207990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb85104246d411aba2069f5aa5f5efb'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.230063 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1413/2782] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:47 2022] DELETE /identity/v3/users/ffb85104246d411aba2069f5aa5f5efb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:48.568909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:48.571239 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.602327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7a918-5aa6-4305-ab65-eb54c8a7fb2b service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:48.708721 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:48.710909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:48.710909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:48.711147 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] PATH_INFO: `/v3/users/c9c4d961baad40648c787c3bbe8c3d87` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:48.712811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c4d961baad40648c787c3bbe8c3d87)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:48.715188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:48.774405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:48.776846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.795529 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c4d961baad40648c787c3bbe8c3d87'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.811428 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7d400-1a07-45a6-a03f-dce22716b006 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:48.819461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:48.820741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:48.821075 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:48.821479 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] PATH_INFO: `/v3/projects/6010e68df762421696f450cd85e89d78` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:48.821823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c4d961baad40648c787c3bbe8c3d87'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.822780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6010e68df762421696f450cd85e89d78)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:48.825406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:48.830113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c4d961baad40648c787c3bbe8c3d87'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.831018 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c4d961baad40648c787c3bbe8c3d87'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.842156 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-03b94724-94f8-45e3-83cc-029c2b5dcff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c4d961baad40648c787c3bbe8c3d87'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.854030 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1370/2783] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:48 2022] DELETE /identity/v3/users/c9c4d961baad40648c787c3bbe8c3d87 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:48.855771 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:48.856064 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:48.856375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:48.878191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:48.879641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.889032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6010e68df762421696f450cd85e89d78', 'request_id': 'req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.896773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6010e68df762421696f450cd85e89d78', 'request_id': 'req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.907908 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6010e68df762421696f450cd85e89d78', 'request_id': 'req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.919418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] The token cache is being invalidate because project 6010e68df762421696f450cd85e89d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:48.919716 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.920724 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:48.922551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1414/2784] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:48 2022] DELETE /identity/v3/projects/6010e68df762421696f450cd85e89d78 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:48.926581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:48.928842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:48.953113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1371/2785] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:48.963545 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:48.965074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:49.001247 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edd8af-9b50-4fc5-bc03-120cf1f7aa96 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:49.006874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:49.008062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.008376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.008659 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] PATH_INFO: `/v3/users/5a351f6f84fe4f4fa979784761373e16` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.010835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a351f6f84fe4f4fa979784761373e16)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:49.012796 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:49.033888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b8ae-322f-453e-970e-d00ccb3e65b5 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:49.038199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:49.038867 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.039152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.039445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.053013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:49.054651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:49.071053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a351f6f84fe4f4fa979784761373e16'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.085640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1372/2786] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:49.091962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a351f6f84fe4f4fa979784761373e16'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.093032 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a351f6f84fe4f4fa979784761373e16'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.093839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a351f6f84fe4f4fa979784761373e16'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.110153 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f27b1025-68d3-4cf7-8150-36a14d2a14d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a351f6f84fe4f4fa979784761373e16'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.140390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1415/2787] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:29:48 2022] DELETE /identity/v3/users/5a351f6f84fe4f4fa979784761373e16 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:49.584501 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:49.586160 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:49.603435 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ed3b7-93c0-4ac5-af9b-40f2c4107d19 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:49.685155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:49.686203 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.686482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.686761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] PATH_INFO: `/v3/projects/18a0553b514544e084db3af0c1523085` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.688108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a0553b514544e084db3af0c1523085)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:49.690860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:49.749399 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a0553b514544e084db3af0c1523085', 'request_id': 'req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.756737 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a0553b514544e084db3af0c1523085', 'request_id': 'req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.766585 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a0553b514544e084db3af0c1523085', 'request_id': 'req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.778026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] The token cache is being invalidate because project 18a0553b514544e084db3af0c1523085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:49.778392 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.779257 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:49.781028 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1373/2788] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:49 2022] DELETE /identity/v3/projects/18a0553b514544e084db3af0c1523085 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:49.785935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aec13b-d096-4353-a686-af4738b29155 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.786474 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aec13b-d096-4353-a686-af4738b29155 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.786867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aec13b-d096-4353-a686-af4738b29155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.806899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-50aec13b-d096-4353-a686-af4738b29155 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:49.808091 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-50aec13b-d096-4353-a686-af4738b29155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:49.884758 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:49.886446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:49.897691 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1416/2789] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:49.909388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.909676 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.909962 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.913284 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8bfbd-f09c-4218-8345-d9c6c4fd4683 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:49.921248 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:49.922592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.922878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.923161 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] PATH_INFO: `/v3/projects/47ecc23a20cd43a081a6a29aa2599278` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:49.925108 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ecc23a20cd43a081a6a29aa2599278)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:49.928546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:49.942546 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:49.944116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:49.970405 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1417/2790] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:49.975297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:49.989238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e2f33-ff1d-4c59-afc0-ff5bf26fd695 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:49.997749 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:49.998554 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:49.998940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:49.999323 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.000336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:50.000643 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.002374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.020364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1418/2791] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:29:50.024616 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.027750 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ecc23a20cd43a081a6a29aa2599278', 'request_id': 'req-84686c95-6ce1-409f-b7c7-75fa0e44caee'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:50.036103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ecc23a20cd43a081a6a29aa2599278', 'request_id': 'req-84686c95-6ce1-409f-b7c7-75fa0e44caee'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:50.043192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798deab-9d10-498c-933a-19088f02cd54 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.050714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ecc23a20cd43a081a6a29aa2599278', 'request_id': 'req-84686c95-6ce1-409f-b7c7-75fa0e44caee'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:50.088125 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] The token cache is being invalidate because project 47ecc23a20cd43a081a6a29aa2599278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:29:50.088838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:50.089549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-84686c95-6ce1-409f-b7c7-75fa0e44caee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:29:50.093507 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1374/2792] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:29:49 2022] DELETE /identity/v3/projects/47ecc23a20cd43a081a6a29aa2599278 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:50.099701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0f9e-66fb-4616-b09d-a418a5c313d6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.099964 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0f9e-66fb-4616-b09d-a418a5c313d6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.100358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0f9e-66fb-4616-b09d-a418a5c313d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.127017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-dcfe0f9e-66fb-4616-b09d-a418a5c313d6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:50.129096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-dcfe0f9e-66fb-4616-b09d-a418a5c313d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.152055 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.152816 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.153106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.153839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.154561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.156605 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.201496 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1419/2793] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:50.205839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.207979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.214471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1375/2794] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:50.215950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.216242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.216540 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.238681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:50.240047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.315189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec6474-bfd5-4a31-b734-281cf5a3a5bf admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.321747 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.322486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.322702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.322990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.323670 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.328659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1376/2795] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:50.330358 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.334051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.336925 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.364777 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1420/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:50.366619 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.380483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ce2ad-c2b8-4e37-a1bd-21158ec17680 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.385883 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.386727 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.387110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.387477 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.388548 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:29:50.389699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.391941 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.407783 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1421/2797] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:29:50.410839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.424206 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b63615-a96f-4e0d-a067-16b8207fe29e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.432047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.433990 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.434451 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.434882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.436070 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.438057 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.438794 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc082d-314b-46df-970a-dec5577167f4 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.446188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.446860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.447143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.447426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.449322 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.451008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1422/2798] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:50.452071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.462717 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de9455-f573-405b-8a65-f4382dd2179b admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.462907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.464702 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.467898 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.468611 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.469156 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.469313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.470016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.472099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.475983 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1377/2799] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:50.485855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.498609 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86488da3-9bdf-4d0f-9789-0a321da89fcb service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.504151 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.504866 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.505150 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.505427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.506170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.507877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.508442 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1423/2800] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:50.510519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.527989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1378/2801] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:50.532860 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.546533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f859-540c-4d49-935b-53bc2a62c516 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.585004 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcf5c7-6f1f-4ce1-ba01-fa6de306227c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.589854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.590539 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.590821 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.591106 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.591942 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.594061 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.615946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.616896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.617183 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.617547 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] PATH_INFO: `/v3/projects/9180a0eff640492fa14647821d87240f/users/3cb8509be15748569c6dd7e048958aa7/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.630593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1424/2802] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:50.631629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.637972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9180a0eff640492fa14647821d87240f, user_id=3cb8509be15748569c6dd7e048958aa7, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.641139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.641828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e88c71-2750-4909-8ab3-36aa1d9cc46e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.646946 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.647622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.647900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.648240 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.649062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.650726 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f26c-cafb-4796-977d-ee3e0a795e6d admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.658920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1379/2803] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:50 2022] PUT /identity/v3/projects/9180a0eff640492fa14647821d87240f/users/3cb8509be15748569c6dd7e048958aa7/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:50.661104 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1425/2804] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:50.664919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.669335 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.669797 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.670130 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.702221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] MFA Rules not processed for user `3cb8509be15748569c6dd7e048958aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:50.703528 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.745181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cece9-3d22-4283-87b6-3ef0362b4c51 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.752105 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.753132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.753593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.753965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.754749 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.756479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.769979 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1380/2805] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:50.771040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.771762 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1426/2806] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:50.775772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.781199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff3a6d-9f1f-4ace-9bc0-791d4c4d346c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.786160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf11cc8-5144-43c5-ac2f-9558314ed558 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.786303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.787247 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.787616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.787965 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] PATH_INFO: `/v3/projects/f1f7081f289e40e0a3bfdd8078c18a31/users/abd82e1ccb114fdeab05375fc4c2e2f9/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.790604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.791254 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.791529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.791800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.792519 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.794420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17879f-af05-425b-b5b3-8f6bbe9e7b21 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.803912 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f7081f289e40e0a3bfdd8078c18a31, user_id=abd82e1ccb114fdeab05375fc4c2e2f9, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.806603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.821471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1381/2807] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:50 2022] PUT /identity/v3/projects/f1f7081f289e40e0a3bfdd8078c18a31/users/abd82e1ccb114fdeab05375fc4c2e2f9/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:50.822457 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1427/2808] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:29:50.830821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.831655 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.831952 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.832236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.858969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] MFA Rules not processed for user `abd82e1ccb114fdeab05375fc4c2e2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:50.860207 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.894452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30976e-6208-4e09-b828-2fc6f6359328 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.922270 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1428/2809] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:29:50.934372 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.936163 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:50.949809 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.950573 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.951025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.951430 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.952084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.954083 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:50.978883 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1382/2810] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:50 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:50.979962 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:50.990096 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536c02-aff5-4804-bcab-241cb487d967 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.993142 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bb94c-ad43-400a-8c39-7cca4a67c211 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:50.995372 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.996025 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.996349 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.996626 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.996789 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:50.997347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:50.997448 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:50.997696 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:50.997973 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:50.998945 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.006165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.007581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.008847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1383/2811] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:51.012801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.016723 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1429/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:51.024301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e75898-f9bb-42b8-b566-16aae76b92bb admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.029027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.029681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.030038 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.030233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.031041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.032557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511ec8-2338-4a68-a9ff-f9f7867aa239 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.042934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1384/2813] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:29:51.047168 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.058175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47deb3-3191-4ef9-addc-651bf8d473ed service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.063011 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.063891 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.064205 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.064498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] PATH_INFO: `/v3/projects/8c5424846f364d83a122063159237905/users/448aac4635ac4f59bc16bed05c0cadde/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.079529 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5424846f364d83a122063159237905, user_id=448aac4635ac4f59bc16bed05c0cadde, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.082230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.095912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1430/2814] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:51 2022] PUT /identity/v3/projects/8c5424846f364d83a122063159237905/users/448aac4635ac4f59bc16bed05c0cadde/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:51.107391 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.107761 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.108099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.135315 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] MFA Rules not processed for user `448aac4635ac4f59bc16bed05c0cadde`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:51.136618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:51.203586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1385/2815] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:51.208301 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.274882 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac10984-e88d-496f-be07-e59c6930d940 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.279586 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.280306 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.280567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.280845 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.281537 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.283462 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.320280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1431/2816] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:29:51 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:51.324432 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.336115 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12be04a-50b5-4e51-8235-050cbac56a5e None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.390915 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.391587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.391869 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.392196 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.392927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.394881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.415842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1386/2817] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:29:51 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:29:51.420013 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.430680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cae606-4293-4de9-a211-bfea3efdd4fb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.436975 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.437642 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.437924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.438197 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.438926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.440927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.451953 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1432/2818] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:51.456578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.471491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108793f8-edef-488c-9ae8-702bb11aaae0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.477482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.478366 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.478650 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.478920 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] PATH_INFO: `/v3/projects/4848177399724e708fdcba3706a09440/users/2a258044323b4a2bbe812d7ed0937492/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.496479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4848177399724e708fdcba3706a09440, user_id=2a258044323b4a2bbe812d7ed0937492, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.499128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.514898 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1387/2819] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:51 2022] PUT /identity/v3/projects/4848177399724e708fdcba3706a09440/users/2a258044323b4a2bbe812d7ed0937492/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:29:51.519336 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.529935 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4452a15-deb3-4394-8442-790808ecfc0c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.588961 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.589622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.589906 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.590181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.590896 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.592510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c08e3-cca2-46d3-8b41-49b531d8bcaf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.602899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1433/2820] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:29:51.608526 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.626266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc9534-b193-4ae2-932c-cda00a99ce04 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.633328 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.634242 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.634523 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.634781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] PATH_INFO: `/v3/projects/4848177399724e708fdcba3706a09440/users/2a258044323b4a2bbe812d7ed0937492/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.637059 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4848177399724e708fdcba3706a09440, user_id=2a258044323b4a2bbe812d7ed0937492, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.639682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.651756 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1388/2821] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:29:51 2022] PUT /identity/v3/projects/4848177399724e708fdcba3706a09440/users/2a258044323b4a2bbe812d7ed0937492/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:29:51.662223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.662509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.662780 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.691154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] MFA Rules not processed for user `2a258044323b4a2bbe812d7ed0937492`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:29:51.692376 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:51.763164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1434/2822] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:29:51.775063 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:51.831568 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526729-5730-4456-aaa4-1fa3a0e317de admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:51.835575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:51.836218 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:51.836525 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:51.836803 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:51.893097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:51.894661 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:51.903437 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1389/2823] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:54.757056 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:54.833049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85863945-0804-4cb7-b9a8-318e6338a1b6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:54.838465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:54.839121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:54.839392 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:54.839678 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:54.841467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:54.853316 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:54.854892 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:54.865189 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1435/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:29:55.479175 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:55.480770 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:29:55.494822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401005-95cb-4b2e-808c-2d0ae716cf88 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:55.498769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:55.499599 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:55.499934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:55.500278 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:55.565827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:55.568355 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:55.578387 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1390/2825] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:29:57.721146 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:29:57.730497 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dc5e6-a4e6-44e6-8361-45ca0a5128a9 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:29:57.738400 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:29:57.739050 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:29:57.739455 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:29:57.739734 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:29:57.760414 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:29:57.761963 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390dc15-724a-41ff-8ad9-2311e308e336 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:29:57.776107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1436/2826] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:29:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:00.937379 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:01.006439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299622-e548-4059-bdb4-621f5620dc66 service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:01.012185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:01.013100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:01.013385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:01.013659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] PATH_INFO: `/v3/users/abd82e1ccb114fdeab05375fc4c2e2f9` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:01.014960 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd82e1ccb114fdeab05375fc4c2e2f9)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:01.016944 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:01.064974 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd82e1ccb114fdeab05375fc4c2e2f9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.078385 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd82e1ccb114fdeab05375fc4c2e2f9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.085021 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd82e1ccb114fdeab05375fc4c2e2f9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.085923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd82e1ccb114fdeab05375fc4c2e2f9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.094193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd82e1ccb114fdeab05375fc4c2e2f9'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.100565 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1391/2827] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:00 2022] DELETE /identity/v3/users/abd82e1ccb114fdeab05375fc4c2e2f9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:01.105569 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:01.106108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:01.106191 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:01.125420 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:01.127096 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:01.210313 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1437/2828] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:01.226099 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:01.228376 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:01.310188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24f917-faef-4bfa-b6f8-a920ab2241c8 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:01.310188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:01.310188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:01.310188 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:01.310484 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:01.323497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:01.325682 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86af3a-64da-40bc-ac26-e3df45735717 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:01.338558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1392/2829] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:01.903983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:01.906218 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:01.922741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdd49a-2595-429c-a96d-5e3f2ad0a7a5 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:01.928339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:01.929581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:01.930069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:01.930359 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] PATH_INFO: `/v3/projects/f1f7081f289e40e0a3bfdd8078c18a31` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:01.931715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f7081f289e40e0a3bfdd8078c18a31)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:01.933921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:01.984382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f7081f289e40e0a3bfdd8078c18a31', 'request_id': 'req-f2a31c41-b4e7-4b57-8e15-cddb1872c446'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:01.990900 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f7081f289e40e0a3bfdd8078c18a31', 'request_id': 'req-f2a31c41-b4e7-4b57-8e15-cddb1872c446'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:02.000098 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f7081f289e40e0a3bfdd8078c18a31', 'request_id': 'req-f2a31c41-b4e7-4b57-8e15-cddb1872c446'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:02.010381 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] The token cache is being invalidate because project f1f7081f289e40e0a3bfdd8078c18a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:02.010672 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:02.011297 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-f2a31c41-b4e7-4b57-8e15-cddb1872c446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:02.012945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1438/2830] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:01 2022] DELETE /identity/v3/projects/f1f7081f289e40e0a3bfdd8078c18a31 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:30:02.017969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.018428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.018512 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.037195 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:02.038664 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:02.104473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1393/2831] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:02.113458 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.113964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.113964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.133936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:02.135157 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:02.143785 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1439/2832] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:02.148167 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.158454 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d971f34-4b99-4233-8de2-c303231f6b8a None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.164382 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.165133 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.165416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.165663 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.166425 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:30:02.166730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.168367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.180327 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1394/2833] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:30:02.185296 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.197877 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b42c8a-7ff1-43b8-b842-dee3f7a4a333 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.206123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.207107 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.207461 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.207860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.208931 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.212402 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.243053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1440/2834] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:30:02 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:02.245723 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.256558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6588-4d16-4740-aad3-686ffdca4630 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.332640 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.333324 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.333603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.334179 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.334886 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.336888 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.360931 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1395/2835] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:30:02 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:02.365314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.376388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f37ec-5c83-4690-8920-ec069a748122 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.381998 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.382671 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.382951 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.383384 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.384285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.386129 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.398065 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1441/2836] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:02.402754 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.413998 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb7aad-720f-4ed9-aa92-0a8fab2835bf admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.419013 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.419827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.420113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.420419 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.421154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.422751 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfed885-1ca8-4786-87a9-963909cdec8e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.438678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1396/2837] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:30:02.439739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.451853 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e1b2-4d0a-4610-8a96-4ef92cd95cfb admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.456849 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.457741 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.458016 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.458286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] PATH_INFO: `/v3/projects/ef034a4bf8264fccae4ad64bfd2d2e15/users/ec8c227efbb042fa8015e321fc459d69/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.475760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef034a4bf8264fccae4ad64bfd2d2e15, user_id=ec8c227efbb042fa8015e321fc459d69, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.478842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.496489 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1442/2838] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:02 2022] PUT /identity/v3/projects/ef034a4bf8264fccae4ad64bfd2d2e15/users/ec8c227efbb042fa8015e321fc459d69/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:30:02.508741 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.509334 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.509618 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.540681 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] MFA Rules not processed for user `ec8c227efbb042fa8015e321fc459d69`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:02.541877 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:02.612281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1397/2839] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:02.636118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:02.637856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:02.704475 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64ccd-1d0a-4260-b8cb-76d51b3a362a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:02.708175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:02.708868 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:02.709154 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:02.709429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:02.718501 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:02.719927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c668e7-dea3-49a0-b5a8-990ab363bf45 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:02.730498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1443/2840] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:08.038841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:08.040397 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:08.111658 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad28c16-2e54-492a-b2a1-72bfc64ad9a8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:08.116537 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:08.117490 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:08.117674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:08.117950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] PATH_INFO: `/v3/users/3cb8509be15748569c6dd7e048958aa7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:08.119142 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb8509be15748569c6dd7e048958aa7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:08.121201 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:08.171303 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb8509be15748569c6dd7e048958aa7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.182955 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb8509be15748569c6dd7e048958aa7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.190774 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb8509be15748569c6dd7e048958aa7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.191505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb8509be15748569c6dd7e048958aa7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.201162 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb8509be15748569c6dd7e048958aa7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.213031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1398/2841] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:08 2022] DELETE /identity/v3/users/3cb8509be15748569c6dd7e048958aa7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:08.218435 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:08.218927 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:08.219126 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:08.243329 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:08.245264 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:08.327876 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1444/2842] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:08.334987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:08.336636 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:08.401036 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a39d-8d60-45eb-8935-890017477d72 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:08.407191 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:08.408006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:08.408316 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:08.408601 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:08.422061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:08.424369 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:08.436300 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1399/2843] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:30:08.769563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:08.771346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:08.785443 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82371a-d6bb-4655-a3ee-44242247e151 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:08.791008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:08.792024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:08.792513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:08.792513 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] PATH_INFO: `/v3/projects/9180a0eff640492fa14647821d87240f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:08.793867 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9180a0eff640492fa14647821d87240f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:08.795920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:08.901388 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9180a0eff640492fa14647821d87240f', 'request_id': 'req-21af4249-6afc-4740-b1a2-7dcc95ebab0c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.916944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9180a0eff640492fa14647821d87240f', 'request_id': 'req-21af4249-6afc-4740-b1a2-7dcc95ebab0c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.925790 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9180a0eff640492fa14647821d87240f', 'request_id': 'req-21af4249-6afc-4740-b1a2-7dcc95ebab0c'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.937150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] The token cache is being invalidate because project 9180a0eff640492fa14647821d87240f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:08.937557 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.938269 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:08.940190 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1445/2844] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:08 2022] DELETE /identity/v3/projects/9180a0eff640492fa14647821d87240f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:08.945434 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:08.955423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c5ea-f094-4163-966e-3e1ed4940b8a service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:09.011699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:09.012616 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:09.012896 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:09.013166 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] PATH_INFO: `/v3/users/448aac4635ac4f59bc16bed05c0cadde` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:09.015093 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] RBAC: Authorizing `identity:delete_user(user_id=448aac4635ac4f59bc16bed05c0cadde)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:09.017069 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:09.065769 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448aac4635ac4f59bc16bed05c0cadde'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.080619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448aac4635ac4f59bc16bed05c0cadde'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.087558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448aac4635ac4f59bc16bed05c0cadde'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.088367 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448aac4635ac4f59bc16bed05c0cadde'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.096428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448aac4635ac4f59bc16bed05c0cadde'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.102281 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1400/2845] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:08 2022] DELETE /identity/v3/users/448aac4635ac4f59bc16bed05c0cadde => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:30:09.405099 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:09.406592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:09.423419 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4249-6afc-4740-b1a2-7dcc95ebab0c admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:09.480992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:09.481948 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:09.482292 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:09.482804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] PATH_INFO: `/v3/projects/8c5424846f364d83a122063159237905` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:09.484465 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5424846f364d83a122063159237905)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:09.486832 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:09.535312 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5424846f364d83a122063159237905', 'request_id': 'req-15e6f75c-e26e-43e7-9f51-e71e56ef3120'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.542196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5424846f364d83a122063159237905', 'request_id': 'req-15e6f75c-e26e-43e7-9f51-e71e56ef3120'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.550095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5424846f364d83a122063159237905', 'request_id': 'req-15e6f75c-e26e-43e7-9f51-e71e56ef3120'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.561233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] The token cache is being invalidate because project 8c5424846f364d83a122063159237905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:09.561518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.562164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.563795 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1446/2846] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:09 2022] DELETE /identity/v3/projects/8c5424846f364d83a122063159237905 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:09.567926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:09.572555 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:09.585757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397a74b-14b7-4d10-b91e-cceb2d4c072e admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:09.655139 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:09.656128 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:09.656580 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:09.656911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] PATH_INFO: `/v3/users/2a258044323b4a2bbe812d7ed0937492` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:09.658193 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a258044323b4a2bbe812d7ed0937492)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:09.660357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:09.708433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a258044323b4a2bbe812d7ed0937492'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.720113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a258044323b4a2bbe812d7ed0937492'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.726699 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a258044323b4a2bbe812d7ed0937492'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.727354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a258044323b4a2bbe812d7ed0937492'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.735423 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-0da7d520-247c-4d08-b72d-253e78a78b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a258044323b4a2bbe812d7ed0937492'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:09.741726 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1401/2847] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:09 2022] DELETE /identity/v3/users/2a258044323b4a2bbe812d7ed0937492 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:10.028764 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:10.030487 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:10.044899 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6f75c-e26e-43e7-9f51-e71e56ef3120 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:10.111008 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:10.111837 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:10.112116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:10.112440 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] PATH_INFO: `/v3/projects/4848177399724e708fdcba3706a09440` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:10.113639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4848177399724e708fdcba3706a09440)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:10.115567 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:10.161869 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4848177399724e708fdcba3706a09440', 'request_id': 'req-6e58bdf4-16b2-472f-9fb6-a31834414e82'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:10.168192 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4848177399724e708fdcba3706a09440', 'request_id': 'req-6e58bdf4-16b2-472f-9fb6-a31834414e82'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:10.177555 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4848177399724e708fdcba3706a09440', 'request_id': 'req-6e58bdf4-16b2-472f-9fb6-a31834414e82'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:10.187902 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] The token cache is being invalidate because project 4848177399724e708fdcba3706a09440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:10.188120 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:10.188775 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:10.190326 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1447/2848] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:10 2022] DELETE /identity/v3/projects/4848177399724e708fdcba3706a09440 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:12.911814 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aae5fe-037c-40c7-80d7-12e6626cc1b3 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:12.912354 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aae5fe-037c-40c7-80d7-12e6626cc1b3 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:12.912439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aae5fe-037c-40c7-80d7-12e6626cc1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:12.930465 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-b8aae5fe-037c-40c7-80d7-12e6626cc1b3 None None] MFA Rules not processed for user `df7cfb1f18484852bb066354d5395dac`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:12.931719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-b8aae5fe-037c-40c7-80d7-12e6626cc1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:12.954069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:12.955532 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:12.992491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1402/2849] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:13.030175 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58bdf4-16b2-472f-9fb6-a31834414e82 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:13.035185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:13.036059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.036365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.036627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] PATH_INFO: `/v3/users/eaaa64eac1e14af89e2ceb7099d62025` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:13.037929 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaa64eac1e14af89e2ceb7099d62025)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:13.039910 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:13.092518 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaa64eac1e14af89e2ceb7099d62025'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.104150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaa64eac1e14af89e2ceb7099d62025'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.105131 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaa64eac1e14af89e2ceb7099d62025'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.105820 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaa64eac1e14af89e2ceb7099d62025'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.112680 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaa64eac1e14af89e2ceb7099d62025'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.127277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1448/2850] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:12 2022] DELETE /identity/v3/users/eaaa64eac1e14af89e2ceb7099d62025 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:30:13.131804 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.132117 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.132441 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:13.151841 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:13.153075 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:13.212904 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1403/2851] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:30:13.223708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:13.225221 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:13.285018 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8de8d-7f1c-434a-bb7d-28c4f0a60082 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:13.288918 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:13.289571 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.289846 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.290118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:13.299793 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:13.301325 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2ae8-af4f-4b45-b37e-57f7d9fad99d service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:13.311566 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1449/2852] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:13.719102 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:13.721185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:13.747561 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58a25b-9fe1-46d0-ab0c-9c6fbcd02217 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:13.753138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:13.754410 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.754913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.755396 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] PATH_INFO: `/v3/projects/e8c0df7abdbe4ce7a78c0bd09da861fe` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:13.756820 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c0df7abdbe4ce7a78c0bd09da861fe)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:13.759081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:13.841713 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c0df7abdbe4ce7a78c0bd09da861fe', 'request_id': 'req-3bf4913c-80ce-4daa-9080-0832de47387c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.849916 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c0df7abdbe4ce7a78c0bd09da861fe', 'request_id': 'req-3bf4913c-80ce-4daa-9080-0832de47387c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.860956 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c0df7abdbe4ce7a78c0bd09da861fe', 'request_id': 'req-3bf4913c-80ce-4daa-9080-0832de47387c'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.873669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] The token cache is being invalidate because project e8c0df7abdbe4ce7a78c0bd09da861fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:13.873926 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.874578 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-3bf4913c-80ce-4daa-9080-0832de47387c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:13.876143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1404/2853] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:13 2022] DELETE /identity/v3/projects/e8c0df7abdbe4ce7a78c0bd09da861fe => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:13.881668 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.882105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.882507 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:13.905681 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:13.906904 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:13.985999 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1450/2854] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:13.994173 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:13.994824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:13.994824 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.022131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:14.024427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:14.036094 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1405/2855] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:14.040860 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.059784 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c1b9c-ab35-4b32-9187-e65e94cc35bd None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.064955 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.065658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.065939 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.066307 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.067237 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:30:14.067686 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.069857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.087060 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1451/2856] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 01:30:14.091767 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.104649 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35e056-091c-46c8-b1bb-c2bb0fcd3b4c None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.111084 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.111854 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.112212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.112603 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.113557 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.115574 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.154471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1406/2857] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:30:14 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:30:14.159615 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.171486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff2f3-a683-4230-afbe-bb99b797c5f1 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.259464 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.260226 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.260602 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.261076 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] PATH_INFO: `/v3/users` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.262042 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.264047 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.284439 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1452/2858] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:30:14 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:30:14.288620 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.305314 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58daf5-3fab-43a7-8222-e633291d37c6 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.310524 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.311170 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.311482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.311714 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.312563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.314251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.318740 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.320983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:14.329295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1407/2859] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:14.330347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.340426 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b797b-0958-4ef9-8267-f245a9fd163c admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.345185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.345857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.346143 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.346422 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.347157 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.348802 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.359251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1408/2860] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:30:14.365198 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.376339 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9847a-4768-42c5-8344-30d46b4fb9bc admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.380972 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.381857 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.382146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.382428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] PATH_INFO: `/v3/projects/b96cb97e5e6f480296f35777da104e06/users/b5e1475e9a5d4af28283280901e95017/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.383093 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734599b-3f16-427a-a4bb-9c04f37a4dcf admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.386800 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.387452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.387739 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.388019 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.389872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:14.407660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96cb97e5e6f480296f35777da104e06, user_id=b5e1475e9a5d4af28283280901e95017, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.410380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb9742-a8ad-44f3-a7d2-2efc737f5543 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.427479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1409/2861] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:14 2022] PUT /identity/v3/projects/b96cb97e5e6f480296f35777da104e06/users/b5e1475e9a5d4af28283280901e95017/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:14.440572 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314903-f716-4e1b-9766-104ffe864545 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.440988 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314903-f716-4e1b-9766-104ffe864545 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.441343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314903-f716-4e1b-9766-104ffe864545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.459828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.461327 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.470838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-55314903-f716-4e1b-9766-104ffe864545 None None] MFA Rules not processed for user `b5e1475e9a5d4af28283280901e95017`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:14.472086 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-55314903-f716-4e1b-9766-104ffe864545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:14.472811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1453/2862] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:30:14.542092 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1410/2863] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:14.554418 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:14.556020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:14.611964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816346-f39b-4dd1-9d11-fc0fc4e0c9a5 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:14.660260 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:14.660923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:14.661198 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:14.661469 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:14.669912 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:14.671331 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7abaf-c0de-46be-819e-8b23b63870f4 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:14.679922 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1454/2864] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:25.991675 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314903-f716-4e1b-9766-104ffe864545 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:26.053475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314903-f716-4e1b-9766-104ffe864545 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:26.058746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:26.059655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.059929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.060200 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] PATH_INFO: `/v3/users/b5e1475e9a5d4af28283280901e95017` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.061446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e1475e9a5d4af28283280901e95017)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:26.063400 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:26.110980 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e1475e9a5d4af28283280901e95017'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.127261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e1475e9a5d4af28283280901e95017'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.135272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e1475e9a5d4af28283280901e95017'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.136006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e1475e9a5d4af28283280901e95017'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.145862 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e1475e9a5d4af28283280901e95017'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.152635 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1411/2865] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:25 2022] DELETE /identity/v3/users/b5e1475e9a5d4af28283280901e95017 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:26.157069 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.157545 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.157631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.175162 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:26.176365 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:26.252169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1455/2866] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:26.260317 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:26.261966 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:26.321470 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058e26-f968-42c7-9532-e4abba32d634 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:26.325592 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:26.326271 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.326549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.326821 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.336330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:26.337781 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c41ec-8cc5-4722-b030-527ba9815ea7 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:26.347550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1412/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:26.746339 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:26.748024 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:26.760143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d75785-53f5-494f-b2da-5f30dec02177 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:26.765334 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:26.766223 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.766498 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.766770 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] PATH_INFO: `/v3/projects/b96cb97e5e6f480296f35777da104e06` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.767983 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96cb97e5e6f480296f35777da104e06)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:26.769917 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:26.822407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96cb97e5e6f480296f35777da104e06', 'request_id': 'req-25afe03f-f427-4c30-88c0-a8c9ab1c9897'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.829351 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96cb97e5e6f480296f35777da104e06', 'request_id': 'req-25afe03f-f427-4c30-88c0-a8c9ab1c9897'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.838994 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96cb97e5e6f480296f35777da104e06', 'request_id': 'req-25afe03f-f427-4c30-88c0-a8c9ab1c9897'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.850798 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] The token cache is being invalidate because project b96cb97e5e6f480296f35777da104e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:26.851087 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.851751 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-25afe03f-f427-4c30-88c0-a8c9ab1c9897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:26.853390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1456/2868] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:26 2022] DELETE /identity/v3/projects/b96cb97e5e6f480296f35777da104e06 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:26.858587 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.858878 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.859159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.877171 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:26.878439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:26.951957 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1413/2869] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:26.960814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:26.961236 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:26.961579 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:26.983020 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:26.984349 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:26.993122 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1457/2870] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:26.997975 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.007746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f68619-8953-4c03-b2c6-cf01a23b1ac8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.012326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.012927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.013309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.013576 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.014309 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:30:27.014617 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.016220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.027989 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1414/2871] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:30:27.032377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.041491 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c68dc-77a0-4173-b23f-319364ec3a70 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.046268 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.046919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.047195 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.047468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.048150 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.050062 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.076814 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1458/2872] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:30:27 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:27.081046 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.091009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec368-5f63-499f-8830-48f599fd287a admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.155463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.156217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.156694 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.156933 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.157730 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.159823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.184088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1415/2873] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:30:27 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:30:27.185568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.196799 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115a4b-d5f0-4fbe-986b-a5fb1dfd8eb8 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.201856 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.202526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.202805 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.203077 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.203802 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.205436 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.217294 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1459/2874] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:27.221220 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.232995 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06496aa-cbd1-4d26-a5f7-8286a4ab9f93 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.237986 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.238660 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.238946 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.239228 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.239987 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.241885 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf17b6-bd83-4b7a-83ed-97ae0732ae32 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.252977 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1416/2875] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 01:30:27.257369 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.269102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6cf9b-f0e6-4694-8a86-28a2ad726544 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.274699 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.275648 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.275944 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.276231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] PATH_INFO: `/v3/projects/ba4f950660124a36b384120a967cec4d/users/4795d9899e1f4b26b0cab51575d2f131/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.294649 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4f950660124a36b384120a967cec4d, user_id=4795d9899e1f4b26b0cab51575d2f131, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.297526 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.313248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1460/2876] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:27 2022] PUT /identity/v3/projects/ba4f950660124a36b384120a967cec4d/users/4795d9899e1f4b26b0cab51575d2f131/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:27.319847 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca737f0-6c9d-4b54-989d-5e56108cc1c6 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.320146 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca737f0-6c9d-4b54-989d-5e56108cc1c6 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.320452 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca737f0-6c9d-4b54-989d-5e56108cc1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.341189 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-7ca737f0-6c9d-4b54-989d-5e56108cc1c6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:27.342457 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-7ca737f0-6c9d-4b54-989d-5e56108cc1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:27.409473 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1417/2877] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:27.419593 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:27.421505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:27.480608 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09d137-8b40-4f6d-ac00-c79841b71429 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:27.485343 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:27.486086 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:27.486494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:27.486794 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:27.498494 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:27.500084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:27.511123 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1461/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:32.134348 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.134950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.134950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.170558 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] MFA Rules not processed for user `4795d9899e1f4b26b0cab51575d2f131`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:32.171801 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:32.244464 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1418/2879] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:32.249235 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.261289 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49443f7-059b-4e7d-8bdd-efa68dc8b578 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.266663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.268000 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.268486 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.268914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.269781 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.271698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.307085 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1462/2880] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:30:32 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:32.308279 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.334659 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d233e-005a-4cad-9091-45166f8d70fb None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.404693 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.405389 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.405759 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.406052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.406630 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.410539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.448720 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1419/2881] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:30:32 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 01:30:32.449366 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.460879 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1415-009a-4bd4-a08b-f1b21b37d557 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.466279 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.467116 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.467422 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.467698 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.468505 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.470341 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.483553 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1463/2882] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:32.488040 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.500509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c338ef-2fd1-4336-95a1-b7b9d0b4d312 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.513827 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.515007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.515607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.515607 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] PATH_INFO: `/v3/projects/6c0e029e2de747d89a5e65209a3c8340/users/93a32729a1af489087c97b47cfd2266f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.542838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0e029e2de747d89a5e65209a3c8340, user_id=93a32729a1af489087c97b47cfd2266f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.545780 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.563213 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1420/2883] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:32 2022] PUT /identity/v3/projects/6c0e029e2de747d89a5e65209a3c8340/users/93a32729a1af489087c97b47cfd2266f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:32.567201 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.582078 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70acd40-02e0-4a91-88bf-8680ba16e6c5 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.650169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.651132 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.651423 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.651697 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.652688 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.656663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35aab08-1234-4d44-8d7f-4cc7ea53ade4 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.667547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1464/2884] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:32.671783 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:32.683335 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938d741-2ca8-46e2-a601-81661724aef2 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:32.692744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:32.694678 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:32.695159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:32.695619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] PATH_INFO: `/v3/projects/6c0e029e2de747d89a5e65209a3c8340/users/93a32729a1af489087c97b47cfd2266f/roles/33906b6d96ba49e6a74f107c852f7119` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:32.698215 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0e029e2de747d89a5e65209a3c8340, user_id=93a32729a1af489087c97b47cfd2266f, role_id=33906b6d96ba49e6a74f107c852f7119)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:32.703061 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:32.724531 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1421/2885] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:32 2022] PUT /identity/v3/projects/6c0e029e2de747d89a5e65209a3c8340/users/93a32729a1af489087c97b47cfd2266f/roles/33906b6d96ba49e6a74f107c852f7119 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:38.536512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b74d-ccab-425e-b4b2-1c1dbf106ae5 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:38.537631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b74d-ccab-425e-b4b2-1c1dbf106ae5 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:38.538121 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b74d-ccab-425e-b4b2-1c1dbf106ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:38.568230 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-6616b74d-ccab-425e-b4b2-1c1dbf106ae5 None None] MFA Rules not processed for user `93a32729a1af489087c97b47cfd2266f`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:38.570243 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-6616b74d-ccab-425e-b4b2-1c1dbf106ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:38.666248 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1465/2886] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:38.674937 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:38.676831 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:38.758153 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d7035-c7b7-49b2-8824-0c359d864c89 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:38.766646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:38.767701 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:38.768181 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:38.768670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:38.842442 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:38.844065 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e3416-4fb6-4a94-84ad-e08ccf9816dd service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:38.856285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1422/2887] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:39.834394 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:39.834938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:39.834938 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:39.856773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] MFA Rules not processed for user `4795d9899e1f4b26b0cab51575d2f131`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:39.858328 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:39.865550 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1466/2888] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:46.902374 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:46.902950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:46.902950 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:46.972074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:46.973539 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:47.049669 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1423/2889] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 12 01:30:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:47.102364 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:47.103865 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:47.200654 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a13be-a6c1-464c-8590-b238d7dbd0f9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:47.207742 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:47.208618 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:47.208936 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:47.209213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:47.218210 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:47.220293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70f528-ac48-48b7-962f-9695d662df2f service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:47.232187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1467/2890] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 01:30:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 01:30:57.596755 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:57.598225 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:57.677863 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be379a-bffc-45ef-b687-800746a5bae8 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:57.682691 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:57.683588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:57.683876 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:57.684154 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] PATH_INFO: `/v3/users/ec8c227efbb042fa8015e321fc459d69` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:57.685433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8c227efbb042fa8015e321fc459d69)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:57.687560 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:57.743405 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8c227efbb042fa8015e321fc459d69'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:57.758295 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8c227efbb042fa8015e321fc459d69'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:57.766304 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8c227efbb042fa8015e321fc459d69'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:57.767152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8c227efbb042fa8015e321fc459d69'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:57.776266 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8c227efbb042fa8015e321fc459d69'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:57.788462 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1424/2891] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:30:57 2022] DELETE /identity/v3/users/ec8c227efbb042fa8015e321fc459d69 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:57.793022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:57.793483 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:57.793570 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:57.811604 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:57.812847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:57.875872 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1468/2892] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:30:57.887031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:57.888646 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:57.951922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c931362-c2d4-4955-9bfd-0451277e15fe admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:57.955811 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:57.956556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:57.956839 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:57.957120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:57.967343 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:57.968984 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03ab6f-8556-471c-b546-f595c3696a72 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:57.980923 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1425/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:30:58.287413 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.289165 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:58.302889 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980677df-7224-4138-8c5c-d5f96e51bca6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.315683 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.316715 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.317183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.318029 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] PATH_INFO: `/v3/projects/ef034a4bf8264fccae4ad64bfd2d2e15` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.341572 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef034a4bf8264fccae4ad64bfd2d2e15)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.343559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.397382 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef034a4bf8264fccae4ad64bfd2d2e15', 'request_id': 'req-304254ff-fd21-4d1a-b5f1-cb218076f0ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:58.404467 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef034a4bf8264fccae4ad64bfd2d2e15', 'request_id': 'req-304254ff-fd21-4d1a-b5f1-cb218076f0ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:58.413411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef034a4bf8264fccae4ad64bfd2d2e15', 'request_id': 'req-304254ff-fd21-4d1a-b5f1-cb218076f0ab'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:58.425810 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] The token cache is being invalidate because project ef034a4bf8264fccae4ad64bfd2d2e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:30:58.426095 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:58.426950 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-304254ff-fd21-4d1a-b5f1-cb218076f0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:30:58.428666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1469/2894] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:30:58 2022] DELETE /identity/v3/projects/ef034a4bf8264fccae4ad64bfd2d2e15 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:30:58.433842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.434132 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.434409 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.453674 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:58.455050 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:58.524293 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1426/2895] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:30:58.529411 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.529835 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.529970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.550663 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:58.552041 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:58.563337 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1470/2896] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:58.564797 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.575246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c3186-255d-4741-a550-6d9b7adcc795 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.580185 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.581024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.581446 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.581719 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] PATH_INFO: `/v3/domains` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.582475 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=96825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 01:30:58.582787 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.584712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.600859 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1427/2897] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 12 01:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 01:30:58.602673 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.612807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9e034-5a4b-48cd-8709-078dbf3c2f23 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.618318 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.619017 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.619266 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.619544 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] PATH_INFO: `/v3/projects` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.620238 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.622231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.650367 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1471/2898] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 12 01:30:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:58.655138 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.668456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734892b-a1fa-4291-ba3c-06b872d0cafa admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.730444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.731444 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.731806 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.732159 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] PATH_INFO: `/v3/users` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.732976 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.735047 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.761852 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1428/2899] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 12 01:30:58 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 01:30:58.763186 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.774997 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234125ca-edf9-4401-b408-3fdc4372d842 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.781452 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.782274 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.782576 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.782861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.783829 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.786431 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.803560 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1472/2900] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:58.804978 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.818034 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10033cfb-e2fe-4b20-b6f6-11cfc73e8975 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.825006 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.825748 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.826037 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.826463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.827260 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.829575 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eb2f7-ed43-486b-a563-80d944d297c6 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.845119 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1429/2901] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 12 01:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 01:30:58.848068 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:58.860516 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aa5d8-387e-4b89-88fe-27f05d807e6a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:58.866072 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:58.866969 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] REQUEST_METHOD: `PUT` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.867406 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.867834 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] PATH_INFO: `/v3/projects/d2b3678f78814182a549b2e043e6e1ce/users/40a2e88b4d324c00b4981864c8b809a7/roles/d2a59677d1e9405f8eeb3527dd367536` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.885267 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b3678f78814182a549b2e043e6e1ce, user_id=40a2e88b4d324c00b4981864c8b809a7, role_id=d2a59677d1e9405f8eeb3527dd367536)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:58.888319 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:58.903298 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1473/2902] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 12 01:30:58 2022] PUT /identity/v3/projects/d2b3678f78814182a549b2e043e6e1ce/users/40a2e88b4d324c00b4981864c8b809a7/roles/d2a59677d1e9405f8eeb3527dd367536 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:30:58.914427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217ba0-7450-4463-bcfa-20807c870f4c None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:58.914731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217ba0-7450-4463-bcfa-20807c870f4c None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:58.915012 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59217ba0-7450-4463-bcfa-20807c870f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:58.945017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-59217ba0-7450-4463-bcfa-20807c870f4c None None] MFA Rules not processed for user `40a2e88b4d324c00b4981864c8b809a7`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:30:58.946261 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-59217ba0-7450-4463-bcfa-20807c870f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:59.024333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1430/2903] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:30:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:30:59.037143 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:30:59.039160 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:30:59.109633 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccd648-24ee-4032-8c1b-d770e8bed50a admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:30:59.114314 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:30:59.115169 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:30:59.115482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:30:59.115760 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:30:59.125875 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:30:59.127622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:30:59.137346 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1474/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:30:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:31:07.916533 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] REQUEST_METHOD: `POST` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:07.917031 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:07.917124 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:07.936567 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.auth.core [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] MFA Rules not processed for user `7f17488343744b5cb36683c882d4bf21`. Rule list: `[]` (Enabled: `True`). {{(pid=96825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:31:07.937776 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:08.005252 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1431/2905] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 12 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:31:08.018022 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:08.019574 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:08.091562 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45ffb7-d8f0-4f6d-a88a-8978dcf80dd0 service cinder] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:08.097320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86ce304b33114f83a8fa63a0227aeef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:08.097992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:08.098277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:08.098666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:08.106964 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:08.108559 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774981c-f9dd-46aa-8bec-4ffff2e4350d service swift] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:08.116280 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1475/2906] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 12 01:31:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 01:31:09.598778 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:09.663922 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3147c55-6d2e-4228-b3bc-2666a8e12f97 None None] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:09.670865 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:09.671746 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:09.672019 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:09.672299 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] PATH_INFO: `/v3/users/40a2e88b4d324c00b4981864c8b809a7` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:09.673542 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a2e88b4d324c00b4981864c8b809a7)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:09.675506 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:09.724182 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a2e88b4d324c00b4981864c8b809a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:09.736113 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a2e88b4d324c00b4981864c8b809a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:09.742828 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a2e88b4d324c00b4981864c8b809a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:09.743563 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a2e88b4d324c00b4981864c8b809a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:09.751628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a2e88b4d324c00b4981864c8b809a7'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:09.758415 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1432/2907] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:31:09 2022] DELETE /identity/v3/users/40a2e88b4d324c00b4981864c8b809a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:31:09.764476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:09.764919 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:09.765026 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:09.783105 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] MFA Rules not processed for user `e279c42792694abda4ec83a94b9d0a59`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:31:09.784228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:09.845046 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1476/2908] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:31:09.854870 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:09.856321 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:09.921556 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da85b1-c006-4493-80fb-7faeb31a6d6d admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:09.925357 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:09.926016 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:09.926302 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:09.926571 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:09.936347 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:09.937710 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:09.947518 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1433/2909] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:31:10.051354 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:10.054646 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:10.074200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270c962-bc28-4f27-9f79-7c41f4192ae7 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:10.079261 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:10.080438 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:10.080776 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:10.081128 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] PATH_INFO: `/v3/users/df7cfb1f18484852bb066354d5395dac` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:10.082470 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7cfb1f18484852bb066354d5395dac)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:10.084568 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:10.140429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7cfb1f18484852bb066354d5395dac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.153102 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7cfb1f18484852bb066354d5395dac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.153839 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7cfb1f18484852bb066354d5395dac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.154446 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7cfb1f18484852bb066354d5395dac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.161424 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7cfb1f18484852bb066354d5395dac'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.176049 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1477/2910] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:31:10 2022] DELETE /identity/v3/users/df7cfb1f18484852bb066354d5395dac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:31:10.221246 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:10.222813 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:10.243935 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205521-bbc0-4c56-9507-7aa1f0ddb1d1 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:10.304074 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:10.305087 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:10.305463 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:10.305743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] PATH_INFO: `/v3/projects/d2b3678f78814182a549b2e043e6e1ce` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:10.306934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b3678f78814182a549b2e043e6e1ce)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:10.309007 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:10.358088 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b3678f78814182a549b2e043e6e1ce', 'request_id': 'req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.359108 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:10.360874 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:10.366491 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b3678f78814182a549b2e043e6e1ce', 'request_id': 'req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.375044 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea702a-1999-4d8b-af5b-3d0cfd19c45e admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:10.375273 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b3678f78814182a549b2e043e6e1ce', 'request_id': 'req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.388670 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] The token cache is being invalidate because project d2b3678f78814182a549b2e043e6e1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:31:10.389027 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.389588 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.391417 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1434/2911] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:31:10 2022] DELETE /identity/v3/projects/d2b3678f78814182a549b2e043e6e1ce => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:31:10.446217 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:10.447441 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:10.448084 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:10.448638 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] PATH_INFO: `/v3/projects/20f14b7d2c3a4fd78ecc16256f3aab75` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:10.451058 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f14b7d2c3a4fd78ecc16256f3aab75)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:10.453731 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:10.507653 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f14b7d2c3a4fd78ecc16256f3aab75', 'request_id': 'req-8cf988a8-9f14-4ee2-b738-dd0545b611d7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.515429 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f14b7d2c3a4fd78ecc16256f3aab75', 'request_id': 'req-8cf988a8-9f14-4ee2-b738-dd0545b611d7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.524804 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f14b7d2c3a4fd78ecc16256f3aab75', 'request_id': 'req-8cf988a8-9f14-4ee2-b738-dd0545b611d7'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.535708 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] The token cache is being invalidate because project 20f14b7d2c3a4fd78ecc16256f3aab75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:31:10.535992 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.536685 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:31:10.538272 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1478/2912] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:31:10 2022] DELETE /identity/v3/projects/20f14b7d2c3a4fd78ecc16256f3aab75 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:31:29.430873 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:31:29.432589 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:29.502137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1457b8b-bbc4-4ffa-9c98-68ecaf5472a1 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:31:29.556504 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:31:29.557152 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:31:29.557439 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:31:29.557712 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:31:29.560009 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:31:29.630739 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:31:29.632199 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:31:29.642244 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1435/2913] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:31:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:32:28.692702 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:32:28.694395 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:32:28.786773 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf988a8-9f14-4ee2-b738-dd0545b611d7 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:32:28.792707 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:32:28.793358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:32:28.793639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:32:28.793914 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:32:28.795943 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:32:28.807965 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:32:28.809623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa157f5-858a-410b-be00-61205a99b82e service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:32:28.820630 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1479/2914] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:32:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:32:56.561107 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:32:56.563126 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:32:56.575479 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fcab-6533-48e4-bfa3-28568db7cc57 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:32:56.579485 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:32:56.580116 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:32:56.580445 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:32:56.580715 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:32:56.582586 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:32:56.682528 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:32:56.684285 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:32:56.705823 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1436/2915] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:32:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:33:00.225140 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:33:00.225627 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:33:00.225923 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:33:00.244410 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] MFA Rules not processed for user `4b84b7192b13421286d86ba3a43d0475`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:33:00.245639 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:33:00.305211 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1480/2916] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:33:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 01:34:04.057109 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:34:04.066842 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89d766-9b52-4900-9329-1ef7db287047 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:34:04.071788 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:34:04.072482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:34:04.072927 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:34:04.073212 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:34:04.075356 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:34:04.085846 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:34:04.087861 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:34:04.095909 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1437/2917] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:34:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:34:04.211111 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:34:04.231182 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8825907-f798-4919-87c0-ae23cec2f1b6 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:34:04.235842 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:34:04.236658 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:34:04.236945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:34:04.237252 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:34:04.239606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:34:04.252133 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:34:04.253656 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:34:04.262481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1481/2918] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:34:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:34:04.373131 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:34:04.382403 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166920-9269-418d-bf14-17ea9c08095f service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:34:04.387077 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:34:04.387716 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:34:04.388026 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:34:04.388416 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:34:04.390482 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:34:04.405051 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:34:04.406497 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:34:04.417341 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1438/2919] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:34:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:34:04.488234 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:34:04.503855 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52feb5-1fe6-4a2a-b63c-1e2e6e3db725 service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:34:04.514377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:34:04.515358 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:34:04.515880 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:34:04.516374 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:34:04.519183 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:34:04.532866 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:34:04.535320 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c343ab7-b427-4c0d-bcdc-017a860d0517 service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:34:04.552847 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1482/2920] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:34:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:10.696641 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:10.698427 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:10.763579 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a19df-25a8-4482-ad16-291b7668eebc service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:10.768052 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00e31520c5b421c8575fc063b1e6f74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:10.768752 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:10.769033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:10.769305 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:10.771492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:10.783155 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:10.784637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:10.793836 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1439/2921] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:35:20.857970 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] REQUEST_METHOD: `POST` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:20.858512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:20.858512 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:20.879353 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.auth.core [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] MFA Rules not processed for user `4b84b7192b13421286d86ba3a43d0475`. Rule list: `[]` (Enabled: `True`). {{(pid=96826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 01:35:20.880551 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:20.889038 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1483/2922] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 12 01:35:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 01:35:23.395081 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:23.396637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:23.408969 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3d78-30a9-40b1-a91a-12232b83bfbc service cinder] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:23.412550 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:23.413233 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:23.413513 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:23.413911 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:23.415838 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:23.426637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:23.428103 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:23.436928 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1440/2923] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:24.124581 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:24.127468 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:24.152677 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16a324-9766-4ddb-8b25-0d13482a97e9 None None] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:24.159482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:24.160425 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:24.160712 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:24.160988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:24.163053 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:24.177315 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:24.181945 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:24.194687 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1484/2924] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:35.907941 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:35.920326 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fa10f-a524-4276-88db-721c77666093 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:35.926418 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:35.927066 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:35.927346 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:35.927619 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:35.929453 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:35.942509 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:35.944112 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:35.954731 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1441/2925] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:38.470689 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:38.487476 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde5bf-6cfd-417a-adbc-ae7daa4c9482 service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:38.492988 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:38.493622 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:38.493909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:38.494176 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:38.496277 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:38.563355 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:38.564828 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:38.574861 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1485/2926] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:51.428654 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:51.440041 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f232bea-4a3c-4237-ac1f-37f907adce92 service nova] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:51.444456 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:51.445595 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:51.445881 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:51.446158 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:51.448097 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:51.524672 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:51.526333 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:51.537822 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1442/2927] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:35:54.933909 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:54.951854 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb2cd-2583-411b-b155-7a43fda0a00e service neutron] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:54.959430 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736e16d1bc05411faea9c943f930acb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:54.961034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:54.961034 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:54.961481 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:54.963940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:54.978623 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:54.980196 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:54.991180 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1486/2928] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:35:56.512010 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:35:56.522637 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c4e6-20d9-42df-a783-d226ef760f3b service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:35:56.529826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:35:56.530467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:35:56.530744 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:35:56.531017 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:35:56.532892 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:35:56.595024 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:35:56.596492 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:35:56.605963 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1443/2929] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:36:03.212480 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:36:03.213979 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:36:03.227980 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3097a-f847-45f9-8a3e-61cfbc18770d service nova] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:36:03.233187 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:36:03.234110 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:36:03.234390 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:36:03.234666 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] PATH_INFO: `/v3/users/4795d9899e1f4b26b0cab51575d2f131` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:36:03.235940 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4795d9899e1f4b26b0cab51575d2f131)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:36:03.237921 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:36:03.290987 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4795d9899e1f4b26b0cab51575d2f131'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.304293 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4795d9899e1f4b26b0cab51575d2f131'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.305202 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4795d9899e1f4b26b0cab51575d2f131'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.305890 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4795d9899e1f4b26b0cab51575d2f131'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.314010 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4795d9899e1f4b26b0cab51575d2f131'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.329285 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1487/2930] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:36:03 2022] DELETE /identity/v3/users/4795d9899e1f4b26b0cab51575d2f131 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:36:03.510165 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:36:03.522644 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614ae1b-6c46-45a8-9aed-807f0d8eeaee service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:36:03.586172 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:36:03.588274 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:36:03.588546 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:36:03.588855 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] PATH_INFO: `/v3/projects/ba4f950660124a36b384120a967cec4d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:36:03.590696 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4f950660124a36b384120a967cec4d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:36:03.593655 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:36:03.648275 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4f950660124a36b384120a967cec4d', 'request_id': 'req-abbdb683-259c-4b00-8d82-2f367b53d5d0'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.654953 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4f950660124a36b384120a967cec4d', 'request_id': 'req-abbdb683-259c-4b00-8d82-2f367b53d5d0'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.663134 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4f950660124a36b384120a967cec4d', 'request_id': 'req-abbdb683-259c-4b00-8d82-2f367b53d5d0'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.673628 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] The token cache is being invalidate because project ba4f950660124a36b384120a967cec4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:36:03.673913 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.674593 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.676276 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1444/2931] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:36:03 2022] DELETE /identity/v3/projects/ba4f950660124a36b384120a967cec4d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:36:03.680920 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:36:03.682510 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:36:03.696563 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ba227-e0f3-46ab-95e1-ac32d62dc135 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:36:03.753485 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:36:03.754350 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:36:03.754626 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:36:03.754928 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] PATH_INFO: `/v3/users/93a32729a1af489087c97b47cfd2266f` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:36:03.756631 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a32729a1af489087c97b47cfd2266f)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:36:03.758947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:36:03.809059 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a32729a1af489087c97b47cfd2266f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.822407 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a32729a1af489087c97b47cfd2266f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.823185 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a32729a1af489087c97b47cfd2266f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.823924 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a32729a1af489087c97b47cfd2266f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.831213 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a32729a1af489087c97b47cfd2266f'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:03.851233 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1488/2932] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:36:03 2022] DELETE /identity/v3/users/93a32729a1af489087c97b47cfd2266f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:36:04.050810 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:36:04.052272 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:36:04.065929 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdb683-259c-4b00-8d82-2f367b53d5d0 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:36:04.125204 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:36:04.126100 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:36:04.126380 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:36:04.126651 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] PATH_INFO: `/v3/projects/6c0e029e2de747d89a5e65209a3c8340` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:36:04.127973 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0e029e2de747d89a5e65209a3c8340)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:36:04.129932 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:36:04.177467 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0e029e2de747d89a5e65209a3c8340', 'request_id': 'req-d739875a-41fc-4238-bac4-dced736bf71f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:04.184000 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0e029e2de747d89a5e65209a3c8340', 'request_id': 'req-d739875a-41fc-4238-bac4-dced736bf71f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:04.194934 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0e029e2de747d89a5e65209a3c8340', 'request_id': 'req-d739875a-41fc-4238-bac4-dced736bf71f'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:04.206623 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] The token cache is being invalidate because project 6c0e029e2de747d89a5e65209a3c8340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:36:04.206907 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:04.207549 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:36:04.209186 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1445/2933] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:36:04 2022] DELETE /identity/v3/projects/6c0e029e2de747d89a5e65209a3c8340 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:37:34.030692 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:34.032200 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:34.136222 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15f2c-e2d3-4676-a68d-0b6bdc963276 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:34.206454 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:34.207118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:34.207403 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:34.207684 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:34.209857 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:34.222926 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:34.225521 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:34.239232 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1489/2934] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:37:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 01:37:44.773711 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:44.775227 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:44.853919 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739875a-41fc-4238-bac4-dced736bf71f admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:44.860527 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '692dbeda127948b3812e6153241e7ce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:44.861522 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] REQUEST_METHOD: `GET` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:44.861917 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:44.862251 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:44.864390 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:44.968856 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:44.970301 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:44.994981 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1446/2935] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 01:37:51.826547 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:51.828089 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:51.859978 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc07ee-bef0-4adc-b811-e76b59420d0d service placement] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:51.864629 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:51.865509 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:51.865788 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:51.866071 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] PATH_INFO: `/v3/users/4b84b7192b13421286d86ba3a43d0475` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:51.867231 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b84b7192b13421286d86ba3a43d0475)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:51.869181 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:51.915719 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b84b7192b13421286d86ba3a43d0475'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:51.927283 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b84b7192b13421286d86ba3a43d0475'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:51.927947 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b84b7192b13421286d86ba3a43d0475'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:51.928561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b84b7192b13421286d86ba3a43d0475'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:51.935377 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b84b7192b13421286d86ba3a43d0475'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:51.950303 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1490/2936] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:37:51 2022] DELETE /identity/v3/users/4b84b7192b13421286d86ba3a43d0475 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:37:52.136330 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:52.155375 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a58958-bc53-4835-9359-df3ac0dc47b8 service neutron] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:52.213612 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:52.214483 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:52.214762 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:52.215033 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] PATH_INFO: `/v3/projects/d5bc257e76d54d0abc67c39792122b0d` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:52.216194 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bc257e76d54d0abc67c39792122b0d)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:52.218137 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:52.265505 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bc257e76d54d0abc67c39792122b0d', 'request_id': 'req-e705e9ac-6035-43e2-ac86-9db3d623abe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.271743 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bc257e76d54d0abc67c39792122b0d', 'request_id': 'req-e705e9ac-6035-43e2-ac86-9db3d623abe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.281008 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bc257e76d54d0abc67c39792122b0d', 'request_id': 'req-e705e9ac-6035-43e2-ac86-9db3d623abe4'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.296032 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] The token cache is being invalidate because project d5bc257e76d54d0abc67c39792122b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:37:52.296328 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.297210 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.299120 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1447/2937] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:37:52 2022] DELETE /identity/v3/projects/d5bc257e76d54d0abc67c39792122b0d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:37:52.303606 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:52.305118 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:52.319492 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6549d9-f636-41d1-b976-7c373c81e6b8 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:52.377592 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:52.378482 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:52.378767 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:52.379045 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] PATH_INFO: `/v3/users/62854276e03e4f91b2589c906c9c5240` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:52.380278 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62854276e03e4f91b2589c906c9c5240)` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:52.382273 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:52.430456 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62854276e03e4f91b2589c906c9c5240'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.443164 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62854276e03e4f91b2589c906c9c5240'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.443901 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62854276e03e4f91b2589c906c9c5240'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.444625 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62854276e03e4f91b2589c906c9c5240'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.452124 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.notifications [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62854276e03e4f91b2589c906c9c5240'} {{(pid=96826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.466811 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1491/2938] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 12 01:37:52 2022] DELETE /identity/v3/users/62854276e03e4f91b2589c906c9c5240 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 01:37:52.644553 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Authenticating user token {{(pid=96825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:37:52.646135 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.fernet_utils [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:37:52.660478 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705e9ac-6035-43e2-ac86-9db3d623abe4 admin admin] Validating token access rules against request {{(pid=96825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:37:52.720049 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e279c42792694abda4ec83a94b9d0a59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '137a089466c04c34b8c9ec48a39d84b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:37:52.721005 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:37:52.721289 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:37:52.721826 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] PATH_INFO: `/v3/projects/7bb154c009f64a7eb99123226bb8cb4f` {{(pid=96825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:37:52.723433 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb154c009f64a7eb99123226bb8cb4f)` {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:37:52.725428 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] RBAC: Authorization granted {{(pid=96825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:37:52.778342 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb154c009f64a7eb99123226bb8cb4f', 'request_id': 'req-53a92a54-86a3-43fd-8e6f-65621e869d44'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.784732 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb154c009f64a7eb99123226bb8cb4f', 'request_id': 'req-53a92a54-86a3-43fd-8e6f-65621e869d44'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.793217 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb154c009f64a7eb99123226bb8cb4f', 'request_id': 'req-53a92a54-86a3-43fd-8e6f-65621e869d44'} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.804471 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] The token cache is being invalidate because project 7bb154c009f64a7eb99123226bb8cb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=96825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 01:37:52.804757 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.805384 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: DEBUG keystone.notifications [None req-53a92a54-86a3-43fd-8e6f-65621e869d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=96825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 01:37:52.806952 dv-cinder-830901-5-iscsi devstack@keystone.service[96825]: [pid: 96825|app: 0|req: 1448/2939] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 12 01:37:52 2022] DELETE /identity/v3/projects/7bb154c009f64a7eb99123226bb8cb4f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 01:42:37.652227 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Authenticating user token {{(pid=96826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 01:42:37.654119 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:42:37.756081 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532819ff-08b7-46ef-9c34-16aa6c6bf8d7 admin admin] Validating token access rules against request {{(pid=96826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 01:42:37.823286 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7dc2677d5c74bc7ad7770c3575f2b74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d005aee3baa40c2af159f881af910d0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=96826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 01:42:37.823966 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] REQUEST_METHOD: `GET` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 01:42:37.824228 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] SCRIPT_NAME: `/identity` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 01:42:37.824561 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=96826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 01:42:37.826807 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.fernet_utils [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=96826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 01:42:37.839311 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 01:42:37.841772 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c21a7-d35a-4dd9-b972-6487aa16715e service placement] RBAC: Authorization granted {{(pid=96826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 01:42:37.852862 dv-cinder-830901-5-iscsi devstack@keystone.service[96826]: [pid: 96826|app: 0|req: 1492/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 12 01:42:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)